<!DOCTYPE html>

<html>
<head>
  <title>lexer.coffee</title>
  <meta http-equiv="content-type" content="text/html; charset=UTF-8">
  <meta name="viewport" content="width=device-width, target-densitydpi=160dpi, initial-scale=1.0; maximum-scale=1.0; user-scalable=0;">
  <link rel="stylesheet" media="all" href="docco.css" />
</head>
<body>
  <div id="container">
    <div id="background"></div>
    
      <ul id="jump_to">
        <li>
          <a class="large" href="javascript:void(0);">Jump To &hellip;</a>
          <a class="small" href="javascript:void(0);">+</a>
          <div id="jump_wrapper">
          <div id="jump_page_wrapper">
            <div id="jump_page">
              
                
                <a class="source" href="browser.html">
                  browser.coffee
                </a>
              
                
                <a class="source" href="cake.html">
                  cake.coffee
                </a>
              
                
                <a class="source" href="coffeescript.html">
                  coffeescript.coffee
                </a>
              
                
                <a class="source" href="command.html">
                  command.coffee
                </a>
              
                
                <a class="source" href="grammar.html">
                  grammar.coffee
                </a>
              
                
                <a class="source" href="helpers.html">
                  helpers.coffee
                </a>
              
                
                <a class="source" href="index.html">
                  index.coffee
                </a>
              
                
                <a class="source" href="lexer.html">
                  lexer.coffee
                </a>
              
                
                <a class="source" href="nodes.html">
                  nodes.coffee
                </a>
              
                
                <a class="source" href="optparse.html">
                  optparse.coffee
                </a>
              
                
                <a class="source" href="register.html">
                  register.coffee
                </a>
              
                
                <a class="source" href="repl.html">
                  repl.coffee
                </a>
              
                
                <a class="source" href="rewriter.html">
                  rewriter.coffee
                </a>
              
                
                <a class="source" href="scope.html">
                  scope.litcoffee
                </a>
              
                
                <a class="source" href="sourcemap.html">
                  sourcemap.litcoffee
                </a>
              
            </div>
          </div>
        </li>
      </ul>
    
    <ul class="sections">
        
          <li id="title">
              <div class="annotation">
                  <h1>lexer.coffee</h1>
              </div>
          </li>
        
        
        
        <li id="section-1">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-1">&#x00a7;</a>
              </div>
              <p>The CoffeeScript Lexer. Uses a series of token-matching regexes to attempt
matches against the beginning of the source code. When a match is found,
a token is produced, we consume the match, and start again. Tokens are in the
form:</p>
<pre><code>[tag, value, locationData]
</code></pre>
<p>where locationData is {first_line, first_column, last_line, last_column, last_line_exclusive, last_column_exclusive}, which is a
format that can be fed directly into <a href="https://github.com/zaach/jison">Jison</a>.  These
are read by jison in the <code>parser.lexer</code> function defined in coffeescript.coffee.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>
{Rewriter, INVERSES, UNFINISHED} = <span class="hljs-built_in">require</span> <span class="hljs-string">&#x27;./rewriter&#x27;</span></pre></div></div>
            
        </li>
        
        
        <li id="section-2">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-2">&#x00a7;</a>
              </div>
              <p>Import the helpers we need.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>{count, starts, compact, repeat, invertLiterate, merge,
attachCommentsToNode, locationDataToString, throwSyntaxError
replaceUnicodeCodePointEscapes, flatten, parseNumber} = <span class="hljs-built_in">require</span> <span class="hljs-string">&#x27;./helpers&#x27;</span></pre></div></div>
            
        </li>
        
        
        <li id="section-3">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-3">&#x00a7;</a>
              </div>
              <h2 id="the-lexer-class">The Lexer Class</h2>

            </div>
            
        </li>
        
        
        <li id="section-4">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-4">&#x00a7;</a>
              </div>
              
            </div>
            
        </li>
        
        
        <li id="section-5">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-5">&#x00a7;</a>
              </div>
              <p>The Lexer class reads a stream of CoffeeScript and divvies it up into tagged
tokens. Some potential ambiguity in the grammar has been avoided by
pushing some extra smarts into the Lexer.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre><span class="hljs-built_in">exports</span>.Lexer = <span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">Lexer</span></span></pre></div></div>
            
        </li>
        
        
        <li id="section-6">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-6">&#x00a7;</a>
              </div>
              <p><strong>tokenize</strong> is the Lexer’s main method. Scan by attempting to match tokens
one at a time, using a regular expression anchored at the start of the
remaining code, or a custom recursive token-matching method
(for interpolations). When the next token has been recorded, we move forward
within the code past the token, and begin again.</p>
<p>Each tokenizing method is responsible for returning the number of characters
it has consumed.</p>
<p>Before returning the token stream, run it through the <a href="rewriter.html">Rewriter</a>.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>  tokenize: <span class="hljs-function"><span class="hljs-params">(code, opts = {})</span> -&gt;</span>
    @literate   = opts.literate  <span class="hljs-comment"># Are we lexing literate CoffeeScript?</span>
    @indent     = <span class="hljs-number">0</span>              <span class="hljs-comment"># The current indentation level.</span>
    @baseIndent = <span class="hljs-number">0</span>              <span class="hljs-comment"># The overall minimum indentation level.</span>
    @continuationLineAdditionalIndent = <span class="hljs-number">0</span> <span class="hljs-comment"># The over-indentation at the current level.</span>
    @outdebt    = <span class="hljs-number">0</span>              <span class="hljs-comment"># The under-outdentation at the current level.</span>
    @indents    = []             <span class="hljs-comment"># The stack of all current indentation levels.</span>
    @indentLiteral = <span class="hljs-string">&#x27;&#x27;</span>          <span class="hljs-comment"># The indentation.</span>
    @ends       = []             <span class="hljs-comment"># The stack for pairing up tokens.</span>
    @tokens     = []             <span class="hljs-comment"># Stream of parsed tokens in the form `[&#x27;TYPE&#x27;, value, location data]`.</span>
    @seenFor    = <span class="hljs-literal">no</span>             <span class="hljs-comment"># Used to recognize `FORIN`, `FOROF` and `FORFROM` tokens.</span>
    @seenImport = <span class="hljs-literal">no</span>             <span class="hljs-comment"># Used to recognize `IMPORT FROM? AS?` tokens.</span>
    @seenExport = <span class="hljs-literal">no</span>             <span class="hljs-comment"># Used to recognize `EXPORT FROM? AS?` tokens.</span>
    @importSpecifierList = <span class="hljs-literal">no</span>    <span class="hljs-comment"># Used to identify when in an `IMPORT {...} FROM? ...`.</span>
    @exportSpecifierList = <span class="hljs-literal">no</span>    <span class="hljs-comment"># Used to identify when in an `EXPORT {...} FROM? ...`.</span>
    @jsxDepth = <span class="hljs-number">0</span>                <span class="hljs-comment"># Used to optimize JSX checks, how deep in JSX we are.</span>
    @jsxObjAttribute = {}        <span class="hljs-comment"># Used to detect if JSX attributes is wrapped in {} (&lt;div {props...} /&gt;).</span>

    @chunkLine =
      opts.line <span class="hljs-keyword">or</span> <span class="hljs-number">0</span>             <span class="hljs-comment"># The start line for the current @chunk.</span>
    @chunkColumn =
      opts.column <span class="hljs-keyword">or</span> <span class="hljs-number">0</span>           <span class="hljs-comment"># The start column of the current @chunk.</span>
    @chunkOffset =
      opts.offset <span class="hljs-keyword">or</span> <span class="hljs-number">0</span>           <span class="hljs-comment"># The start offset for the current @chunk.</span>
    @locationDataCompensations =
      opts.locationDataCompensations <span class="hljs-keyword">or</span> {} <span class="hljs-comment"># The location data compensations for the current @chunk.</span>
    code = @clean code           <span class="hljs-comment"># The stripped, cleaned original source code.</span></pre></div></div>
            
        </li>
        
        
        <li id="section-7">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-7">&#x00a7;</a>
              </div>
              <p>At every position, run through this list of attempted matches,
short-circuiting if any of them succeed. Their order determines precedence:
<code>@literalToken</code> is the fallback catch-all.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>    i = <span class="hljs-number">0</span>
    <span class="hljs-keyword">while</span> @chunk = code[i..]
      consumed = \
           @identifierToken() <span class="hljs-keyword">or</span>
           @commentToken()    <span class="hljs-keyword">or</span>
           @whitespaceToken() <span class="hljs-keyword">or</span>
           @lineToken()       <span class="hljs-keyword">or</span>
           @stringToken()     <span class="hljs-keyword">or</span>
           @numberToken()     <span class="hljs-keyword">or</span>
           @jsxToken()        <span class="hljs-keyword">or</span>
           @regexToken()      <span class="hljs-keyword">or</span>
           @jsToken()         <span class="hljs-keyword">or</span>
           @literalToken()</pre></div></div>
            
        </li>
        
        
        <li id="section-8">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-8">&#x00a7;</a>
              </div>
              <p>Update position.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>      [@chunkLine, @chunkColumn, @chunkOffset] = @getLineAndColumnFromChunk consumed

      i += consumed

      <span class="hljs-keyword">return</span> {@tokens, index: i} <span class="hljs-keyword">if</span> opts.untilBalanced <span class="hljs-keyword">and</span> @ends.length <span class="hljs-keyword">is</span> <span class="hljs-number">0</span>

    @closeIndentation()
    @error <span class="hljs-string">&quot;missing <span class="hljs-subst">#{end.tag}</span>&quot;</span>, (end.origin ? end)[<span class="hljs-number">2</span>] <span class="hljs-keyword">if</span> end = @ends.pop()
    <span class="hljs-keyword">return</span> @tokens <span class="hljs-keyword">if</span> opts.rewrite <span class="hljs-keyword">is</span> <span class="hljs-literal">off</span>
    (<span class="hljs-keyword">new</span> Rewriter).rewrite @tokens</pre></div></div>
            
        </li>
        
        
        <li id="section-9">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-9">&#x00a7;</a>
              </div>
              <p>Preprocess the code to remove leading and trailing whitespace, carriage
returns, etc. If we’re lexing literate CoffeeScript, strip external Markdown
by removing all lines that aren’t indented by at least four spaces or a tab.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>  clean: <span class="hljs-function"><span class="hljs-params">(code)</span> -&gt;</span>
    thusFar = <span class="hljs-number">0</span>
    <span class="hljs-keyword">if</span> code.charCodeAt(<span class="hljs-number">0</span>) <span class="hljs-keyword">is</span> BOM
      code = code.slice <span class="hljs-number">1</span>
      @locationDataCompensations[<span class="hljs-number">0</span>] = <span class="hljs-number">1</span>
      thusFar += <span class="hljs-number">1</span>
    <span class="hljs-keyword">if</span> WHITESPACE.test code
      code = <span class="hljs-string">&quot;\n<span class="hljs-subst">#{code}</span>&quot;</span>
      @chunkLine--
      @locationDataCompensations[<span class="hljs-number">0</span>] ?= <span class="hljs-number">0</span>
      @locationDataCompensations[<span class="hljs-number">0</span>] -= <span class="hljs-number">1</span>
    code = code
      .replace <span class="hljs-regexp">/\r/g</span>, <span class="hljs-function"><span class="hljs-params">(match, offset)</span> =&gt;</span>
        @locationDataCompensations[thusFar + offset] = <span class="hljs-number">1</span>
        <span class="hljs-string">&#x27;&#x27;</span>
      .replace TRAILING_SPACES, <span class="hljs-string">&#x27;&#x27;</span>
    code = invertLiterate code <span class="hljs-keyword">if</span> @literate
    code</pre></div></div>
            
        </li>
        
        
        <li id="section-10">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-10">&#x00a7;</a>
              </div>
              <h2 id="tokenizers">Tokenizers</h2>

            </div>
            
        </li>
        
        
        <li id="section-11">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-11">&#x00a7;</a>
              </div>
              
            </div>
            
        </li>
        
        
        <li id="section-12">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-12">&#x00a7;</a>
              </div>
              <p>Matches identifying literals: variables, keywords, method names, etc.
Check to ensure that JavaScript reserved words aren’t being used as
identifiers. Because CoffeeScript reserves a handful of keywords that are
allowed in JavaScript, we’re careful not to tag them as keywords when
referenced as property names here, so you can still do <code>jQuery.is()</code> even
though <code>is</code> means <code>===</code> otherwise.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>  identifierToken: <span class="hljs-function">-&gt;</span>
    inJSXTag = @atJSXTag()
    regex = <span class="hljs-keyword">if</span> inJSXTag <span class="hljs-keyword">then</span> JSX_ATTRIBUTE <span class="hljs-keyword">else</span> IDENTIFIER
    <span class="hljs-keyword">return</span> <span class="hljs-number">0</span> <span class="hljs-keyword">unless</span> match = regex.exec @chunk
    [input, id, colon] = match</pre></div></div>
            
        </li>
        
        
        <li id="section-13">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-13">&#x00a7;</a>
              </div>
              <p>Preserve length of id for location data</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>    idLength = id.length
    poppedToken = <span class="hljs-literal">undefined</span>
    <span class="hljs-keyword">if</span> id <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;own&#x27;</span> <span class="hljs-keyword">and</span> @tag() <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;FOR&#x27;</span>
      @token <span class="hljs-string">&#x27;OWN&#x27;</span>, id
      <span class="hljs-keyword">return</span> id.length
    <span class="hljs-keyword">if</span> id <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;from&#x27;</span> <span class="hljs-keyword">and</span> @tag() <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;YIELD&#x27;</span>
      @token <span class="hljs-string">&#x27;FROM&#x27;</span>, id
      <span class="hljs-keyword">return</span> id.length
    <span class="hljs-keyword">if</span> id <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;as&#x27;</span> <span class="hljs-keyword">and</span> @seenImport
      <span class="hljs-keyword">if</span> @value() <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;*&#x27;</span>
        @tokens[@tokens.length - <span class="hljs-number">1</span>][<span class="hljs-number">0</span>] = <span class="hljs-string">&#x27;IMPORT_ALL&#x27;</span>
      <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> @value(<span class="hljs-literal">yes</span>) <span class="hljs-keyword">in</span> COFFEE_KEYWORDS
        prev = @prev()
        [prev[<span class="hljs-number">0</span>], prev[<span class="hljs-number">1</span>]] = [<span class="hljs-string">&#x27;IDENTIFIER&#x27;</span>, @value(<span class="hljs-literal">yes</span>)]
      <span class="hljs-keyword">if</span> @tag() <span class="hljs-keyword">in</span> [<span class="hljs-string">&#x27;DEFAULT&#x27;</span>, <span class="hljs-string">&#x27;IMPORT_ALL&#x27;</span>, <span class="hljs-string">&#x27;IDENTIFIER&#x27;</span>]
        @token <span class="hljs-string">&#x27;AS&#x27;</span>, id
        <span class="hljs-keyword">return</span> id.length
    <span class="hljs-keyword">if</span> id <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;as&#x27;</span> <span class="hljs-keyword">and</span> @seenExport
      <span class="hljs-keyword">if</span> @tag() <span class="hljs-keyword">in</span> [<span class="hljs-string">&#x27;IDENTIFIER&#x27;</span>, <span class="hljs-string">&#x27;DEFAULT&#x27;</span>]
        @token <span class="hljs-string">&#x27;AS&#x27;</span>, id
        <span class="hljs-keyword">return</span> id.length
      <span class="hljs-keyword">if</span> @value(<span class="hljs-literal">yes</span>) <span class="hljs-keyword">in</span> COFFEE_KEYWORDS
        prev = @prev()
        [prev[<span class="hljs-number">0</span>], prev[<span class="hljs-number">1</span>]] = [<span class="hljs-string">&#x27;IDENTIFIER&#x27;</span>, @value(<span class="hljs-literal">yes</span>)]
        @token <span class="hljs-string">&#x27;AS&#x27;</span>, id
        <span class="hljs-keyword">return</span> id.length
    <span class="hljs-keyword">if</span> id <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;default&#x27;</span> <span class="hljs-keyword">and</span> @seenExport <span class="hljs-keyword">and</span> @tag() <span class="hljs-keyword">in</span> [<span class="hljs-string">&#x27;EXPORT&#x27;</span>, <span class="hljs-string">&#x27;AS&#x27;</span>]
      @token <span class="hljs-string">&#x27;DEFAULT&#x27;</span>, id
      <span class="hljs-keyword">return</span> id.length
    <span class="hljs-keyword">if</span> id <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;assert&#x27;</span> <span class="hljs-keyword">and</span> (@seenImport <span class="hljs-keyword">or</span> @seenExport) <span class="hljs-keyword">and</span> @tag() <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;STRING&#x27;</span>
      @token <span class="hljs-string">&#x27;ASSERT&#x27;</span>, id
      <span class="hljs-keyword">return</span> id.length
    <span class="hljs-keyword">if</span> id <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;do&#x27;</span> <span class="hljs-keyword">and</span> regExSuper = <span class="hljs-regexp">/^(\s*super)(?!\(\))/</span>.exec @chunk[<span class="hljs-number">3.</span>..]
      @token <span class="hljs-string">&#x27;SUPER&#x27;</span>, <span class="hljs-string">&#x27;super&#x27;</span>
      @token <span class="hljs-string">&#x27;CALL_START&#x27;</span>, <span class="hljs-string">&#x27;(&#x27;</span>
      @token <span class="hljs-string">&#x27;CALL_END&#x27;</span>, <span class="hljs-string">&#x27;)&#x27;</span>
      [input, sup] = regExSuper
      <span class="hljs-keyword">return</span> sup.length + <span class="hljs-number">3</span>

    prev = @prev()

    tag =
      <span class="hljs-keyword">if</span> colon <span class="hljs-keyword">or</span> prev? <span class="hljs-keyword">and</span>
         (prev[<span class="hljs-number">0</span>] <span class="hljs-keyword">in</span> [<span class="hljs-string">&#x27;.&#x27;</span>, <span class="hljs-string">&#x27;?.&#x27;</span>, <span class="hljs-string">&#x27;::&#x27;</span>, <span class="hljs-string">&#x27;?::&#x27;</span>] <span class="hljs-keyword">or</span>
         <span class="hljs-keyword">not</span> prev.spaced <span class="hljs-keyword">and</span> prev[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;@&#x27;</span>)
        <span class="hljs-string">&#x27;PROPERTY&#x27;</span>
      <span class="hljs-keyword">else</span>
        <span class="hljs-string">&#x27;IDENTIFIER&#x27;</span>

    tokenData = {}
    <span class="hljs-keyword">if</span> tag <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;IDENTIFIER&#x27;</span> <span class="hljs-keyword">and</span> (id <span class="hljs-keyword">in</span> JS_KEYWORDS <span class="hljs-keyword">or</span> id <span class="hljs-keyword">in</span> COFFEE_KEYWORDS) <span class="hljs-keyword">and</span>
       <span class="hljs-keyword">not</span> (@exportSpecifierList <span class="hljs-keyword">and</span> id <span class="hljs-keyword">in</span> COFFEE_KEYWORDS)
      tag = id.toUpperCase()
      <span class="hljs-keyword">if</span> tag <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;WHEN&#x27;</span> <span class="hljs-keyword">and</span> @tag() <span class="hljs-keyword">in</span> LINE_BREAK
        tag = <span class="hljs-string">&#x27;LEADING_WHEN&#x27;</span>
      <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> tag <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;FOR&#x27;</span>
        @seenFor = {endsLength: @ends.length}
      <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> tag <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;UNLESS&#x27;</span>
        tag = <span class="hljs-string">&#x27;IF&#x27;</span>
      <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> tag <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;IMPORT&#x27;</span>
        @seenImport = <span class="hljs-literal">yes</span>
      <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> tag <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;EXPORT&#x27;</span>
        @seenExport = <span class="hljs-literal">yes</span>
      <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> tag <span class="hljs-keyword">in</span> UNARY
        tag = <span class="hljs-string">&#x27;UNARY&#x27;</span>
      <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> tag <span class="hljs-keyword">in</span> RELATION
        <span class="hljs-keyword">if</span> tag <span class="hljs-keyword">isnt</span> <span class="hljs-string">&#x27;INSTANCEOF&#x27;</span> <span class="hljs-keyword">and</span> @seenFor
          tag = <span class="hljs-string">&#x27;FOR&#x27;</span> + tag
          @seenFor = <span class="hljs-literal">no</span>
        <span class="hljs-keyword">else</span>
          tag = <span class="hljs-string">&#x27;RELATION&#x27;</span>
          <span class="hljs-keyword">if</span> @value() <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;!&#x27;</span>
            poppedToken = @tokens.pop()
            tokenData.invert = poppedToken.data?.original ? poppedToken[<span class="hljs-number">1</span>]
    <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> tag <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;IDENTIFIER&#x27;</span> <span class="hljs-keyword">and</span> @seenFor <span class="hljs-keyword">and</span> id <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;from&#x27;</span> <span class="hljs-keyword">and</span>
       isForFrom(prev)
      tag = <span class="hljs-string">&#x27;FORFROM&#x27;</span>
      @seenFor = <span class="hljs-literal">no</span></pre></div></div>
            
        </li>
        
        
        <li id="section-14">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-14">&#x00a7;</a>
              </div>
              <p>Throw an error on attempts to use <code>get</code> or <code>set</code> as keywords, or
what CoffeeScript would normally interpret as calls to functions named
<code>get</code> or <code>set</code>, i.e. <code>get({foo: function () {}})</code>.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>    <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> tag <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;PROPERTY&#x27;</span> <span class="hljs-keyword">and</span> prev
      <span class="hljs-keyword">if</span> prev.spaced <span class="hljs-keyword">and</span> prev[<span class="hljs-number">0</span>] <span class="hljs-keyword">in</span> CALLABLE <span class="hljs-keyword">and</span> <span class="hljs-regexp">/^[gs]et$/</span>.test(prev[<span class="hljs-number">1</span>]) <span class="hljs-keyword">and</span>
         @tokens.length &gt; <span class="hljs-number">1</span> <span class="hljs-keyword">and</span> @tokens[@tokens.length - <span class="hljs-number">2</span>][<span class="hljs-number">0</span>] <span class="hljs-keyword">not</span> <span class="hljs-keyword">in</span> [<span class="hljs-string">&#x27;.&#x27;</span>, <span class="hljs-string">&#x27;?.&#x27;</span>, <span class="hljs-string">&#x27;@&#x27;</span>]
        @error <span class="hljs-string">&quot;&#x27;<span class="hljs-subst">#{prev[<span class="hljs-number">1</span>]}</span>&#x27; cannot be used as a keyword, or as a function call
        without parentheses&quot;</span>, prev[<span class="hljs-number">2</span>]
      <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> prev[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;.&#x27;</span> <span class="hljs-keyword">and</span> @tokens.length &gt; <span class="hljs-number">1</span> <span class="hljs-keyword">and</span> (prevprev = @tokens[@tokens.length - <span class="hljs-number">2</span>])[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;UNARY&#x27;</span> <span class="hljs-keyword">and</span> prevprev[<span class="hljs-number">1</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;new&#x27;</span>
        prevprev[<span class="hljs-number">0</span>] = <span class="hljs-string">&#x27;NEW_TARGET&#x27;</span>
      <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> prev[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;.&#x27;</span> <span class="hljs-keyword">and</span> @tokens.length &gt; <span class="hljs-number">1</span> <span class="hljs-keyword">and</span> (prevprev = @tokens[@tokens.length - <span class="hljs-number">2</span>])[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;IMPORT&#x27;</span> <span class="hljs-keyword">and</span> prevprev[<span class="hljs-number">1</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;import&#x27;</span>
        @seenImport = <span class="hljs-literal">no</span>
        prevprev[<span class="hljs-number">0</span>] = <span class="hljs-string">&#x27;IMPORT_META&#x27;</span>
      <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> @tokens.length &gt; <span class="hljs-number">2</span>
        prevprev = @tokens[@tokens.length - <span class="hljs-number">2</span>]
        <span class="hljs-keyword">if</span> prev[<span class="hljs-number">0</span>] <span class="hljs-keyword">in</span> [<span class="hljs-string">&#x27;@&#x27;</span>, <span class="hljs-string">&#x27;THIS&#x27;</span>] <span class="hljs-keyword">and</span> prevprev <span class="hljs-keyword">and</span> prevprev.spaced <span class="hljs-keyword">and</span>
           <span class="hljs-regexp">/^[gs]et$/</span>.test(prevprev[<span class="hljs-number">1</span>]) <span class="hljs-keyword">and</span>
           @tokens[@tokens.length - <span class="hljs-number">3</span>][<span class="hljs-number">0</span>] <span class="hljs-keyword">not</span> <span class="hljs-keyword">in</span> [<span class="hljs-string">&#x27;.&#x27;</span>, <span class="hljs-string">&#x27;?.&#x27;</span>, <span class="hljs-string">&#x27;@&#x27;</span>]
          @error <span class="hljs-string">&quot;&#x27;<span class="hljs-subst">#{prevprev[<span class="hljs-number">1</span>]}</span>&#x27; cannot be used as a keyword, or as a
          function call without parentheses&quot;</span>, prevprev[<span class="hljs-number">2</span>]

    <span class="hljs-keyword">if</span> tag <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;IDENTIFIER&#x27;</span> <span class="hljs-keyword">and</span> id <span class="hljs-keyword">in</span> RESERVED <span class="hljs-keyword">and</span> <span class="hljs-keyword">not</span> inJSXTag
      @error <span class="hljs-string">&quot;reserved word &#x27;<span class="hljs-subst">#{id}</span>&#x27;&quot;</span>, length: id.length

    <span class="hljs-keyword">unless</span> tag <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;PROPERTY&#x27;</span> <span class="hljs-keyword">or</span> @exportSpecifierList <span class="hljs-keyword">or</span> @importSpecifierList
      <span class="hljs-keyword">if</span> id <span class="hljs-keyword">in</span> COFFEE_ALIASES
        alias = id
        id = COFFEE_ALIAS_MAP[id]
        tokenData.original = alias
      tag = <span class="hljs-keyword">switch</span> id
        <span class="hljs-keyword">when</span> <span class="hljs-string">&#x27;!&#x27;</span>                 <span class="hljs-keyword">then</span> <span class="hljs-string">&#x27;UNARY&#x27;</span>
        <span class="hljs-keyword">when</span> <span class="hljs-string">&#x27;==&#x27;</span>, <span class="hljs-string">&#x27;!=&#x27;</span>          <span class="hljs-keyword">then</span> <span class="hljs-string">&#x27;COMPARE&#x27;</span>
        <span class="hljs-keyword">when</span> <span class="hljs-string">&#x27;true&#x27;</span>, <span class="hljs-string">&#x27;false&#x27;</span>     <span class="hljs-keyword">then</span> <span class="hljs-string">&#x27;BOOL&#x27;</span>
        <span class="hljs-keyword">when</span> <span class="hljs-string">&#x27;break&#x27;</span>, <span class="hljs-string">&#x27;continue&#x27;</span>, \
             <span class="hljs-string">&#x27;debugger&#x27;</span>          <span class="hljs-keyword">then</span> <span class="hljs-string">&#x27;STATEMENT&#x27;</span>
        <span class="hljs-keyword">when</span> <span class="hljs-string">&#x27;&amp;&amp;&#x27;</span>, <span class="hljs-string">&#x27;||&#x27;</span>          <span class="hljs-keyword">then</span> id
        <span class="hljs-keyword">else</span>  tag

    tagToken = @token tag, id, length: idLength, data: tokenData
    tagToken.origin = [tag, alias, tagToken[<span class="hljs-number">2</span>]] <span class="hljs-keyword">if</span> alias
    <span class="hljs-keyword">if</span> poppedToken
      [tagToken[<span class="hljs-number">2</span>].first_line, tagToken[<span class="hljs-number">2</span>].first_column, tagToken[<span class="hljs-number">2</span>].range[<span class="hljs-number">0</span>]] =
        [poppedToken[<span class="hljs-number">2</span>].first_line, poppedToken[<span class="hljs-number">2</span>].first_column, poppedToken[<span class="hljs-number">2</span>].range[<span class="hljs-number">0</span>]]
    <span class="hljs-keyword">if</span> colon
      colonOffset = input.lastIndexOf <span class="hljs-keyword">if</span> inJSXTag <span class="hljs-keyword">then</span> <span class="hljs-string">&#x27;=&#x27;</span> <span class="hljs-keyword">else</span> <span class="hljs-string">&#x27;:&#x27;</span>
      colonToken = @token <span class="hljs-string">&#x27;:&#x27;</span>, <span class="hljs-string">&#x27;:&#x27;</span>, offset: colonOffset
      colonToken.jsxColon = <span class="hljs-literal">yes</span> <span class="hljs-keyword">if</span> inJSXTag <span class="hljs-comment"># used by rewriter</span>
    <span class="hljs-keyword">if</span> inJSXTag <span class="hljs-keyword">and</span> tag <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;IDENTIFIER&#x27;</span> <span class="hljs-keyword">and</span> prev[<span class="hljs-number">0</span>] <span class="hljs-keyword">isnt</span> <span class="hljs-string">&#x27;:&#x27;</span>
      @token <span class="hljs-string">&#x27;,&#x27;</span>, <span class="hljs-string">&#x27;,&#x27;</span>, length: <span class="hljs-number">0</span>, origin: tagToken, generated: <span class="hljs-literal">yes</span>

    input.length</pre></div></div>
            
        </li>
        
        
        <li id="section-15">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-15">&#x00a7;</a>
              </div>
              <p>Matches numbers, including decimals, hex, and exponential notation.
Be careful not to interfere with ranges in progress.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>  numberToken: <span class="hljs-function">-&gt;</span>
    <span class="hljs-keyword">return</span> <span class="hljs-number">0</span> <span class="hljs-keyword">unless</span> match = NUMBER.exec @chunk

    number = match[<span class="hljs-number">0</span>]
    lexedLength = number.length

    <span class="hljs-keyword">switch</span>
      <span class="hljs-keyword">when</span> <span class="hljs-regexp">/^0[BOX]/</span>.test number
        @error <span class="hljs-string">&quot;radix prefix in &#x27;<span class="hljs-subst">#{number}</span>&#x27; must be lowercase&quot;</span>, offset: <span class="hljs-number">1</span>
      <span class="hljs-keyword">when</span> <span class="hljs-regexp">/^0\d*[89]/</span>.test number
        @error <span class="hljs-string">&quot;decimal literal &#x27;<span class="hljs-subst">#{number}</span>&#x27; must not be prefixed with &#x27;0&#x27;&quot;</span>, length: lexedLength
      <span class="hljs-keyword">when</span> <span class="hljs-regexp">/^0\d+/</span>.test number
        @error <span class="hljs-string">&quot;octal literal &#x27;<span class="hljs-subst">#{number}</span>&#x27; must be prefixed with &#x27;0o&#x27;&quot;</span>, length: lexedLength

    parsedValue = parseNumber number
    tokenData = {parsedValue}

    tag = <span class="hljs-keyword">if</span> parsedValue <span class="hljs-keyword">is</span> <span class="hljs-literal">Infinity</span> <span class="hljs-keyword">then</span> <span class="hljs-string">&#x27;INFINITY&#x27;</span> <span class="hljs-keyword">else</span> <span class="hljs-string">&#x27;NUMBER&#x27;</span>
    <span class="hljs-keyword">if</span> tag <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;INFINITY&#x27;</span>
      tokenData.original = number
    @token tag, number,
      length: lexedLength
      data: tokenData
    lexedLength</pre></div></div>
            
        </li>
        
        
        <li id="section-16">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-16">&#x00a7;</a>
              </div>
              <p>Matches strings, including multiline strings, as well as heredocs, with or without
interpolation.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>  stringToken: <span class="hljs-function">-&gt;</span>
    [quote] = STRING_START.exec(@chunk) || []
    <span class="hljs-keyword">return</span> <span class="hljs-number">0</span> <span class="hljs-keyword">unless</span> quote</pre></div></div>
            
        </li>
        
        
        <li id="section-17">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-17">&#x00a7;</a>
              </div>
              <p>If the preceding token is <code>from</code> and this is an import or export statement,
properly tag the <code>from</code>.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>    prev = @prev()
    <span class="hljs-keyword">if</span> prev <span class="hljs-keyword">and</span> @value() <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;from&#x27;</span> <span class="hljs-keyword">and</span> (@seenImport <span class="hljs-keyword">or</span> @seenExport)
      prev[<span class="hljs-number">0</span>] = <span class="hljs-string">&#x27;FROM&#x27;</span>

    regex = <span class="hljs-keyword">switch</span> quote
      <span class="hljs-keyword">when</span> <span class="hljs-string">&quot;&#x27;&quot;</span>   <span class="hljs-keyword">then</span> STRING_SINGLE
      <span class="hljs-keyword">when</span> <span class="hljs-string">&#x27;&quot;&#x27;</span>   <span class="hljs-keyword">then</span> STRING_DOUBLE
      <span class="hljs-keyword">when</span> <span class="hljs-string">&quot;&#x27;&#x27;&#x27;&quot;</span> <span class="hljs-keyword">then</span> HEREDOC_SINGLE
      <span class="hljs-keyword">when</span> <span class="hljs-string">&#x27;&quot;&quot;&quot;&#x27;</span> <span class="hljs-keyword">then</span> HEREDOC_DOUBLE

    {tokens, index: end} = @matchWithInterpolations regex, quote

    heredoc = quote.length <span class="hljs-keyword">is</span> <span class="hljs-number">3</span>
    <span class="hljs-keyword">if</span> heredoc</pre></div></div>
            
        </li>
        
        
        <li id="section-18">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-18">&#x00a7;</a>
              </div>
              <p>Find the smallest indentation. It will be removed from all lines later.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>      indent = <span class="hljs-literal">null</span>
      doc = (token[<span class="hljs-number">1</span>] <span class="hljs-keyword">for</span> token, i <span class="hljs-keyword">in</span> tokens <span class="hljs-keyword">when</span> token[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;NEOSTRING&#x27;</span>).join <span class="hljs-string">&#x27;#{}&#x27;</span>
      <span class="hljs-keyword">while</span> match = HEREDOC_INDENT.exec doc
        attempt = match[<span class="hljs-number">1</span>]
        indent = attempt <span class="hljs-keyword">if</span> indent <span class="hljs-keyword">is</span> <span class="hljs-literal">null</span> <span class="hljs-keyword">or</span> <span class="hljs-number">0</span> &lt; attempt.length &lt; indent.length

    delimiter = quote.charAt(<span class="hljs-number">0</span>)
    @mergeInterpolationTokens tokens, {quote, indent, endOffset: end}, <span class="hljs-function"><span class="hljs-params">(value)</span> =&gt;</span>
      @validateUnicodeCodePointEscapes value, delimiter: quote

    <span class="hljs-keyword">if</span> @atJSXTag()
      @token <span class="hljs-string">&#x27;,&#x27;</span>, <span class="hljs-string">&#x27;,&#x27;</span>, length: <span class="hljs-number">0</span>, origin: @prev, generated: <span class="hljs-literal">yes</span>

    end</pre></div></div>
            
        </li>
        
        
        <li id="section-19">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-19">&#x00a7;</a>
              </div>
              <p>Matches and consumes comments. The comments are taken out of the token
stream and saved for later, to be reinserted into the output after
everything has been parsed and the JavaScript code generated.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>  commentToken: <span class="hljs-function"><span class="hljs-params">(chunk = @chunk, {heregex, returnCommentTokens = <span class="hljs-literal">no</span>, offsetInChunk = <span class="hljs-number">0</span>} = {})</span> -&gt;</span>
    <span class="hljs-keyword">return</span> <span class="hljs-number">0</span> <span class="hljs-keyword">unless</span> match = chunk.match COMMENT
    [commentWithSurroundingWhitespace, hereLeadingWhitespace, hereComment, hereTrailingWhitespace, lineComment] = match
    contents = <span class="hljs-literal">null</span></pre></div></div>
            
        </li>
        
        
        <li id="section-20">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-20">&#x00a7;</a>
              </div>
              <p>Does this comment follow code on the same line?</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>    leadingNewline = <span class="hljs-regexp">/^\s*\n+\s*#/</span>.test commentWithSurroundingWhitespace
    <span class="hljs-keyword">if</span> hereComment
      matchIllegal = HERECOMMENT_ILLEGAL.exec hereComment
      <span class="hljs-keyword">if</span> matchIllegal
        @error <span class="hljs-string">&quot;block comments cannot contain <span class="hljs-subst">#{matchIllegal[<span class="hljs-number">0</span>]}</span>&quot;</span>,
          offset: <span class="hljs-string">&#x27;###&#x27;</span>.length + matchIllegal.index, length: matchIllegal[<span class="hljs-number">0</span>].length</pre></div></div>
            
        </li>
        
        
        <li id="section-21">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-21">&#x00a7;</a>
              </div>
              <p>Parse indentation or outdentation as if this block comment didn’t exist.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>      chunk = chunk.replace <span class="hljs-string">&quot;###<span class="hljs-subst">#{hereComment}</span>###&quot;</span>, <span class="hljs-string">&#x27;&#x27;</span></pre></div></div>
            
        </li>
        
        
        <li id="section-22">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-22">&#x00a7;</a>
              </div>
              <p>Remove leading newlines, like <code>Rewriter::removeLeadingNewlines</code>, to
avoid the creation of unwanted <code>TERMINATOR</code> tokens.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>      chunk = chunk.replace <span class="hljs-regexp">/^\n+/</span>, <span class="hljs-string">&#x27;&#x27;</span>
      @lineToken {chunk}</pre></div></div>
            
        </li>
        
        
        <li id="section-23">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-23">&#x00a7;</a>
              </div>
              <p>Pull out the ###-style comment’s content, and format it.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>      content = hereComment
      contents = [{
        content
        length: commentWithSurroundingWhitespace.length - hereLeadingWhitespace.length - hereTrailingWhitespace.length
        leadingWhitespace: hereLeadingWhitespace
      }]
    <span class="hljs-keyword">else</span></pre></div></div>
            
        </li>
        
        
        <li id="section-24">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-24">&#x00a7;</a>
              </div>
              <p>The <code>COMMENT</code> regex captures successive line comments as one token.
Remove any leading newlines before the first comment, but preserve
blank lines between line comments.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>      leadingNewlines = <span class="hljs-string">&#x27;&#x27;</span>
      content = lineComment.replace <span class="hljs-regexp">/^(\n*)/</span>, <span class="hljs-function"><span class="hljs-params">(leading)</span> -&gt;</span>
        leadingNewlines = leading
        <span class="hljs-string">&#x27;&#x27;</span>
      precedingNonCommentLines = <span class="hljs-string">&#x27;&#x27;</span>
      hasSeenFirstCommentLine = <span class="hljs-literal">no</span>
      contents =
        content.split <span class="hljs-string">&#x27;\n&#x27;</span>
        .map (line, index) -&gt;
          <span class="hljs-keyword">unless</span> line.indexOf(<span class="hljs-string">&#x27;#&#x27;</span>) &gt; <span class="hljs-number">-1</span>
            precedingNonCommentLines += <span class="hljs-string">&quot;\n<span class="hljs-subst">#{line}</span>&quot;</span>
            <span class="hljs-keyword">return</span>
          leadingWhitespace = <span class="hljs-string">&#x27;&#x27;</span>
          content = line.replace <span class="hljs-regexp">/^([ |\t]*)#/</span>, <span class="hljs-function"><span class="hljs-params">(_, whitespace)</span> -&gt;</span>
            leadingWhitespace = whitespace
            <span class="hljs-string">&#x27;&#x27;</span>
          comment = {
            content
            length: <span class="hljs-string">&#x27;#&#x27;</span>.length + content.length
            leadingWhitespace: <span class="hljs-string">&quot;<span class="hljs-subst">#{<span class="hljs-keyword">unless</span> hasSeenFirstCommentLine <span class="hljs-keyword">then</span> leadingNewlines <span class="hljs-keyword">else</span> <span class="hljs-string">&#x27;&#x27;</span>}</span><span class="hljs-subst">#{precedingNonCommentLines}</span><span class="hljs-subst">#{leadingWhitespace}</span>&quot;</span>
            precededByBlankLine: !!precedingNonCommentLines
          }
          hasSeenFirstCommentLine = <span class="hljs-literal">yes</span>
          precedingNonCommentLines = <span class="hljs-string">&#x27;&#x27;</span>
          comment
        .filter (comment) -&gt; comment
<span class="hljs-function">
    <span class="hljs-title">getIndentSize</span> = <span class="hljs-params">({leadingWhitespace, nonInitial})</span> -&gt;</span>
      lastNewlineIndex = leadingWhitespace.lastIndexOf <span class="hljs-string">&#x27;\n&#x27;</span>
      <span class="hljs-keyword">if</span> hereComment? <span class="hljs-keyword">or</span> <span class="hljs-keyword">not</span> nonInitial
        <span class="hljs-keyword">return</span> <span class="hljs-literal">null</span> <span class="hljs-keyword">unless</span> lastNewlineIndex &gt; <span class="hljs-number">-1</span>
      <span class="hljs-keyword">else</span>
        lastNewlineIndex ?= <span class="hljs-number">-1</span>
      leadingWhitespace.length - <span class="hljs-number">1</span> - lastNewlineIndex
    commentAttachments = <span class="hljs-keyword">for</span> {content, length, leadingWhitespace, precededByBlankLine}, i <span class="hljs-keyword">in</span> contents
      nonInitial = i <span class="hljs-keyword">isnt</span> <span class="hljs-number">0</span>
      leadingNewlineOffset = <span class="hljs-keyword">if</span> nonInitial <span class="hljs-keyword">then</span> <span class="hljs-number">1</span> <span class="hljs-keyword">else</span> <span class="hljs-number">0</span>
      offsetInChunk += leadingNewlineOffset + leadingWhitespace.length
      indentSize = getIndentSize {leadingWhitespace, nonInitial}
      noIndent = <span class="hljs-keyword">not</span> indentSize? <span class="hljs-keyword">or</span> indentSize <span class="hljs-keyword">is</span> <span class="hljs-number">-1</span>
      commentAttachment = {
        content
        here: hereComment?
        newLine: leadingNewline <span class="hljs-keyword">or</span> nonInitial <span class="hljs-comment"># Line comments after the first one start new lines, by definition.</span>
        locationData: @makeLocationData {offsetInChunk, length}
        precededByBlankLine
        indentSize
        indented:  <span class="hljs-keyword">not</span> noIndent <span class="hljs-keyword">and</span> indentSize &gt; @indent
        outdented: <span class="hljs-keyword">not</span> noIndent <span class="hljs-keyword">and</span> indentSize &lt; @indent
      }
      commentAttachment.heregex = <span class="hljs-literal">yes</span> <span class="hljs-keyword">if</span> heregex
      offsetInChunk += length
      commentAttachment

    prev = @prev()
    <span class="hljs-keyword">unless</span> prev</pre></div></div>
            
        </li>
        
        
        <li id="section-25">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-25">&#x00a7;</a>
              </div>
              <p>If there’s no previous token, create a placeholder token to attach
this comment to; and follow with a newline.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>      commentAttachments[<span class="hljs-number">0</span>].newLine = <span class="hljs-literal">yes</span>
      @lineToken chunk: @chunk[commentWithSurroundingWhitespace.length..], offset: commentWithSurroundingWhitespace.length <span class="hljs-comment"># Set the indent.</span>
      placeholderToken = @makeToken <span class="hljs-string">&#x27;JS&#x27;</span>, <span class="hljs-string">&#x27;&#x27;</span>, offset: commentWithSurroundingWhitespace.length, generated: <span class="hljs-literal">yes</span>
      placeholderToken.comments = commentAttachments
      @tokens.push placeholderToken
      @newlineToken commentWithSurroundingWhitespace.length
    <span class="hljs-keyword">else</span>
      attachCommentsToNode commentAttachments, prev

    <span class="hljs-keyword">return</span> commentAttachments <span class="hljs-keyword">if</span> returnCommentTokens
    commentWithSurroundingWhitespace.length</pre></div></div>
            
        </li>
        
        
        <li id="section-26">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-26">&#x00a7;</a>
              </div>
              <p>Matches JavaScript interpolated directly into the source via backticks.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>  jsToken: <span class="hljs-function">-&gt;</span>
    <span class="hljs-keyword">return</span> <span class="hljs-number">0</span> <span class="hljs-keyword">unless</span> @chunk.charAt(<span class="hljs-number">0</span>) <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;`&#x27;</span> <span class="hljs-keyword">and</span>
      (match = (matchedHere = HERE_JSTOKEN.exec(@chunk)) <span class="hljs-keyword">or</span> JSTOKEN.exec(@chunk))</pre></div></div>
            
        </li>
        
        
        <li id="section-27">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-27">&#x00a7;</a>
              </div>
              <p>Convert escaped backticks to backticks, and escaped backslashes
just before escaped backticks to backslashes</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>    script = match[<span class="hljs-number">1</span>]
    {length} = match[<span class="hljs-number">0</span>]
    @token <span class="hljs-string">&#x27;JS&#x27;</span>, script, {length, data: {here: !!matchedHere}}
    length</pre></div></div>
            
        </li>
        
        
        <li id="section-28">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-28">&#x00a7;</a>
              </div>
              <p>Matches regular expression literals, as well as multiline extended ones.
Lexing regular expressions is difficult to distinguish from division, so we
borrow some basic heuristics from JavaScript and Ruby.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>  regexToken: <span class="hljs-function">-&gt;</span>
    <span class="hljs-keyword">switch</span>
      <span class="hljs-keyword">when</span> match = REGEX_ILLEGAL.exec @chunk
        @error <span class="hljs-string">&quot;regular expressions cannot begin with <span class="hljs-subst">#{match[<span class="hljs-number">2</span>]}</span>&quot;</span>,
          offset: match.index + match[<span class="hljs-number">1</span>].length
      <span class="hljs-keyword">when</span> match = @matchWithInterpolations HEREGEX, <span class="hljs-string">&#x27;///&#x27;</span>
        {tokens, index} = match
        comments = []
        <span class="hljs-keyword">while</span> matchedComment = HEREGEX_COMMENT.exec @chunk[<span class="hljs-number">0.</span>..index]
          {index: commentIndex} = matchedComment
          [fullMatch, leadingWhitespace, comment] = matchedComment
          comments.push {comment, offsetInChunk: commentIndex + leadingWhitespace.length}
        commentTokens = flatten(
          <span class="hljs-keyword">for</span> commentOpts <span class="hljs-keyword">in</span> comments
            @commentToken commentOpts.comment, <span class="hljs-built_in">Object</span>.assign commentOpts, heregex: <span class="hljs-literal">yes</span>, returnCommentTokens: <span class="hljs-literal">yes</span>
        )
      <span class="hljs-keyword">when</span> match = REGEX.exec @chunk
        [regex, body, closed] = match
        @validateEscapes body, isRegex: <span class="hljs-literal">yes</span>, offsetInChunk: <span class="hljs-number">1</span>
        index = regex.length
        prev = @prev()
        <span class="hljs-keyword">if</span> prev
          <span class="hljs-keyword">if</span> prev.spaced <span class="hljs-keyword">and</span> prev[<span class="hljs-number">0</span>] <span class="hljs-keyword">in</span> CALLABLE
            <span class="hljs-keyword">return</span> <span class="hljs-number">0</span> <span class="hljs-keyword">if</span> <span class="hljs-keyword">not</span> closed <span class="hljs-keyword">or</span> POSSIBLY_DIVISION.test regex
          <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> prev[<span class="hljs-number">0</span>] <span class="hljs-keyword">in</span> NOT_REGEX
            <span class="hljs-keyword">return</span> <span class="hljs-number">0</span>
        @error <span class="hljs-string">&#x27;missing / (unclosed regex)&#x27;</span> <span class="hljs-keyword">unless</span> closed
      <span class="hljs-keyword">else</span>
        <span class="hljs-keyword">return</span> <span class="hljs-number">0</span>

    [flags] = REGEX_FLAGS.exec @chunk[index..]
    end = index + flags.length
    origin = @makeToken <span class="hljs-string">&#x27;REGEX&#x27;</span>, <span class="hljs-literal">null</span>, length: end
    <span class="hljs-keyword">switch</span>
      <span class="hljs-keyword">when</span> <span class="hljs-keyword">not</span> VALID_FLAGS.test flags
        @error <span class="hljs-string">&quot;invalid regular expression flags <span class="hljs-subst">#{flags}</span>&quot;</span>, offset: index, length: flags.length
      <span class="hljs-keyword">when</span> regex <span class="hljs-keyword">or</span> tokens.length <span class="hljs-keyword">is</span> <span class="hljs-number">1</span>
        delimiter = <span class="hljs-keyword">if</span> body <span class="hljs-keyword">then</span> <span class="hljs-string">&#x27;/&#x27;</span> <span class="hljs-keyword">else</span> <span class="hljs-string">&#x27;///&#x27;</span>
        body ?= tokens[<span class="hljs-number">0</span>][<span class="hljs-number">1</span>]
        @validateUnicodeCodePointEscapes body, {delimiter}
        @token <span class="hljs-string">&#x27;REGEX&#x27;</span>, <span class="hljs-string">&quot;/<span class="hljs-subst">#{body}</span>/<span class="hljs-subst">#{flags}</span>&quot;</span>, {length: end, origin, data: {delimiter}}
      <span class="hljs-keyword">else</span>
        @token <span class="hljs-string">&#x27;REGEX_START&#x27;</span>, <span class="hljs-string">&#x27;(&#x27;</span>,    {length: <span class="hljs-number">0</span>, origin, generated: <span class="hljs-literal">yes</span>}
        @token <span class="hljs-string">&#x27;IDENTIFIER&#x27;</span>, <span class="hljs-string">&#x27;RegExp&#x27;</span>, length: <span class="hljs-number">0</span>, generated: <span class="hljs-literal">yes</span>
        @token <span class="hljs-string">&#x27;CALL_START&#x27;</span>, <span class="hljs-string">&#x27;(&#x27;</span>,      length: <span class="hljs-number">0</span>, generated: <span class="hljs-literal">yes</span>
        @mergeInterpolationTokens tokens, {double: <span class="hljs-literal">yes</span>, heregex: {flags}, endOffset: end - flags.length, quote: <span class="hljs-string">&#x27;///&#x27;</span>}, <span class="hljs-function"><span class="hljs-params">(str)</span> =&gt;</span>
          @validateUnicodeCodePointEscapes str, {delimiter}
        <span class="hljs-keyword">if</span> flags
          @token <span class="hljs-string">&#x27;,&#x27;</span>, <span class="hljs-string">&#x27;,&#x27;</span>,                    offset: index - <span class="hljs-number">1</span>, length: <span class="hljs-number">0</span>, generated: <span class="hljs-literal">yes</span>
          @token <span class="hljs-string">&#x27;STRING&#x27;</span>, <span class="hljs-string">&#x27;&quot;&#x27;</span> + flags + <span class="hljs-string">&#x27;&quot;&#x27;</span>, offset: index,     length: flags.length
        @token <span class="hljs-string">&#x27;)&#x27;</span>, <span class="hljs-string">&#x27;)&#x27;</span>,                      offset: end,       length: <span class="hljs-number">0</span>, generated: <span class="hljs-literal">yes</span>
        @token <span class="hljs-string">&#x27;REGEX_END&#x27;</span>, <span class="hljs-string">&#x27;)&#x27;</span>,              offset: end,       length: <span class="hljs-number">0</span>, generated: <span class="hljs-literal">yes</span></pre></div></div>
            
        </li>
        
        
        <li id="section-29">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-29">&#x00a7;</a>
              </div>
              <p>Explicitly attach any heregex comments to the REGEX/REGEX_END token.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>    <span class="hljs-keyword">if</span> commentTokens?.length
      addTokenData @tokens[@tokens.length - <span class="hljs-number">1</span>],
        heregexCommentTokens: commentTokens

    end</pre></div></div>
            
        </li>
        
        
        <li id="section-30">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-30">&#x00a7;</a>
              </div>
              <p>Matches newlines, indents, and outdents, and determines which is which.
If we can detect that the current line is continued onto the next line,
then the newline is suppressed:</p>
<pre><code>elements
  .each( ... )
  .map( ... )
</code></pre>
<p>Keeps track of the level of indentation, because a single outdent token
can close multiple indents, so we need to know how far in we happen to be.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>  lineToken: <span class="hljs-function"><span class="hljs-params">({chunk = @chunk, offset = <span class="hljs-number">0</span>} = {})</span> -&gt;</span>
    <span class="hljs-keyword">return</span> <span class="hljs-number">0</span> <span class="hljs-keyword">unless</span> match = MULTI_DENT.exec chunk
    indent = match[<span class="hljs-number">0</span>]

    prev = @prev()
    backslash = prev?[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;\\&#x27;</span>
    @seenFor = <span class="hljs-literal">no</span> <span class="hljs-keyword">unless</span> (backslash <span class="hljs-keyword">or</span> @seenFor?.endsLength &lt; @ends.length) <span class="hljs-keyword">and</span> @seenFor
    @seenImport = <span class="hljs-literal">no</span> <span class="hljs-keyword">unless</span> (backslash <span class="hljs-keyword">and</span> @seenImport) <span class="hljs-keyword">or</span> @importSpecifierList
    @seenExport = <span class="hljs-literal">no</span> <span class="hljs-keyword">unless</span> (backslash <span class="hljs-keyword">and</span> @seenExport) <span class="hljs-keyword">or</span> @exportSpecifierList

    size = indent.length - <span class="hljs-number">1</span> - indent.lastIndexOf <span class="hljs-string">&#x27;\n&#x27;</span>
    noNewlines = @unfinished()

    newIndentLiteral = <span class="hljs-keyword">if</span> size &gt; <span class="hljs-number">0</span> <span class="hljs-keyword">then</span> indent[-size..] <span class="hljs-keyword">else</span> <span class="hljs-string">&#x27;&#x27;</span>
    <span class="hljs-keyword">unless</span> <span class="hljs-regexp">/^(.?)\1*$/</span>.exec newIndentLiteral
      @error <span class="hljs-string">&#x27;mixed indentation&#x27;</span>, offset: indent.length
      <span class="hljs-keyword">return</span> indent.length

    minLiteralLength = <span class="hljs-built_in">Math</span>.min newIndentLiteral.length, @indentLiteral.length
    <span class="hljs-keyword">if</span> newIndentLiteral[...minLiteralLength] <span class="hljs-keyword">isnt</span> @indentLiteral[...minLiteralLength]
      @error <span class="hljs-string">&#x27;indentation mismatch&#x27;</span>, offset: indent.length
      <span class="hljs-keyword">return</span> indent.length

    <span class="hljs-keyword">if</span> size - @continuationLineAdditionalIndent <span class="hljs-keyword">is</span> @indent
      <span class="hljs-keyword">if</span> noNewlines <span class="hljs-keyword">then</span> @suppressNewlines() <span class="hljs-keyword">else</span> @newlineToken offset
      <span class="hljs-keyword">return</span> indent.length

    <span class="hljs-keyword">if</span> size &gt; @indent
      <span class="hljs-keyword">if</span> noNewlines
        @continuationLineAdditionalIndent = size - @indent <span class="hljs-keyword">unless</span> backslash
        <span class="hljs-keyword">if</span> @continuationLineAdditionalIndent
          prev.continuationLineIndent = @indent + @continuationLineAdditionalIndent
        @suppressNewlines()
        <span class="hljs-keyword">return</span> indent.length
      <span class="hljs-keyword">unless</span> @tokens.length
        @baseIndent = @indent = size
        @indentLiteral = newIndentLiteral
        <span class="hljs-keyword">return</span> indent.length
      diff = size - @indent + @outdebt
      @token <span class="hljs-string">&#x27;INDENT&#x27;</span>, diff, offset: offset + indent.length - size, length: size
      @indents.push diff
      @ends.push {tag: <span class="hljs-string">&#x27;OUTDENT&#x27;</span>}
      @outdebt = @continuationLineAdditionalIndent = <span class="hljs-number">0</span>
      @indent = size
      @indentLiteral = newIndentLiteral
    <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> size &lt; @baseIndent
      @error <span class="hljs-string">&#x27;missing indentation&#x27;</span>, offset: offset + indent.length
    <span class="hljs-keyword">else</span>
      endsContinuationLineIndentation = @continuationLineAdditionalIndent &gt; <span class="hljs-number">0</span>
      @continuationLineAdditionalIndent = <span class="hljs-number">0</span>
      @outdentToken {moveOut: @indent - size, noNewlines, outdentLength: indent.length, offset, indentSize: size, endsContinuationLineIndentation}
    indent.length</pre></div></div>
            
        </li>
        
        
        <li id="section-31">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-31">&#x00a7;</a>
              </div>
              <p>Record an outdent token or multiple tokens, if we happen to be moving back
inwards past several recorded indents. Sets new @indent value.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>  outdentToken: <span class="hljs-function"><span class="hljs-params">({moveOut, noNewlines, outdentLength = <span class="hljs-number">0</span>, offset = <span class="hljs-number">0</span>, indentSize, endsContinuationLineIndentation})</span> -&gt;</span>
    decreasedIndent = @indent - moveOut
    <span class="hljs-keyword">while</span> moveOut &gt; <span class="hljs-number">0</span>
      lastIndent = @indents[@indents.length - <span class="hljs-number">1</span>]
      <span class="hljs-keyword">if</span> <span class="hljs-keyword">not</span> lastIndent
        @outdebt = moveOut = <span class="hljs-number">0</span>
      <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> @outdebt <span class="hljs-keyword">and</span> moveOut &lt;= @outdebt
        @outdebt -= moveOut
        moveOut   = <span class="hljs-number">0</span>
      <span class="hljs-keyword">else</span>
        dent = @indents.pop() + @outdebt
        <span class="hljs-keyword">if</span> outdentLength <span class="hljs-keyword">and</span> @chunk[outdentLength] <span class="hljs-keyword">in</span> INDENTABLE_CLOSERS
          decreasedIndent -= dent - moveOut
          moveOut = dent
        @outdebt = <span class="hljs-number">0</span></pre></div></div>
            
        </li>
        
        
        <li id="section-32">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-32">&#x00a7;</a>
              </div>
              <p>pair might call outdentToken, so preserve decreasedIndent</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>        @pair <span class="hljs-string">&#x27;OUTDENT&#x27;</span>
        @token <span class="hljs-string">&#x27;OUTDENT&#x27;</span>, moveOut, length: outdentLength, indentSize: indentSize + moveOut - dent
        moveOut -= dent
    @outdebt -= moveOut <span class="hljs-keyword">if</span> dent
    @suppressSemicolons()

    <span class="hljs-keyword">unless</span> @tag() <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;TERMINATOR&#x27;</span> <span class="hljs-keyword">or</span> noNewlines
      terminatorToken = @token <span class="hljs-string">&#x27;TERMINATOR&#x27;</span>, <span class="hljs-string">&#x27;\n&#x27;</span>, offset: offset + outdentLength, length: <span class="hljs-number">0</span>
      terminatorToken.endsContinuationLineIndentation = {preContinuationLineIndent: @indent} <span class="hljs-keyword">if</span> endsContinuationLineIndentation
    @indent = decreasedIndent
    @indentLiteral = @indentLiteral[...decreasedIndent]
    this</pre></div></div>
            
        </li>
        
        
        <li id="section-33">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-33">&#x00a7;</a>
              </div>
              <p>Matches and consumes non-meaningful whitespace. Tag the previous token
as being “spaced”, because there are some cases where it makes a difference.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>  whitespaceToken: <span class="hljs-function">-&gt;</span>
    <span class="hljs-keyword">return</span> <span class="hljs-number">0</span> <span class="hljs-keyword">unless</span> (match = WHITESPACE.exec @chunk) <span class="hljs-keyword">or</span>
                    (nline = @chunk.charAt(<span class="hljs-number">0</span>) <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;\n&#x27;</span>)
    prev = @prev()
    prev[<span class="hljs-keyword">if</span> match <span class="hljs-keyword">then</span> <span class="hljs-string">&#x27;spaced&#x27;</span> <span class="hljs-keyword">else</span> <span class="hljs-string">&#x27;newLine&#x27;</span>] = <span class="hljs-literal">true</span> <span class="hljs-keyword">if</span> prev
    <span class="hljs-keyword">if</span> match <span class="hljs-keyword">then</span> match[<span class="hljs-number">0</span>].length <span class="hljs-keyword">else</span> <span class="hljs-number">0</span></pre></div></div>
            
        </li>
        
        
        <li id="section-34">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-34">&#x00a7;</a>
              </div>
              <p>Generate a newline token. Consecutive newlines get merged together.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>  newlineToken: <span class="hljs-function"><span class="hljs-params">(offset)</span> -&gt;</span>
    @suppressSemicolons()
    @token <span class="hljs-string">&#x27;TERMINATOR&#x27;</span>, <span class="hljs-string">&#x27;\n&#x27;</span>, {offset, length: <span class="hljs-number">0</span>} <span class="hljs-keyword">unless</span> @tag() <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;TERMINATOR&#x27;</span>
    this</pre></div></div>
            
        </li>
        
        
        <li id="section-35">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-35">&#x00a7;</a>
              </div>
              <p>Use a <code>\</code> at a line-ending to suppress the newline.
The slash is removed here once its job is done.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>  suppressNewlines: <span class="hljs-function">-&gt;</span>
    prev = @prev()
    <span class="hljs-keyword">if</span> prev[<span class="hljs-number">1</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;\\&#x27;</span>
      <span class="hljs-keyword">if</span> prev.comments <span class="hljs-keyword">and</span> @tokens.length &gt; <span class="hljs-number">1</span></pre></div></div>
            
        </li>
        
        
        <li id="section-36">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-36">&#x00a7;</a>
              </div>
              <p><code>@tokens.length</code> should be at least 2 (some code, then <code>\</code>).
If something puts a <code>\</code> after nothing, they deserve to lose any
comments that trail it.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>        attachCommentsToNode prev.comments, @tokens[@tokens.length - <span class="hljs-number">2</span>]
      @tokens.pop()
    this

  jsxToken: <span class="hljs-function">-&gt;</span>
    firstChar = @chunk[<span class="hljs-number">0</span>]</pre></div></div>
            
        </li>
        
        
        <li id="section-37">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-37">&#x00a7;</a>
              </div>
              <p>Check the previous token to detect if attribute is spread.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>    prevChar = <span class="hljs-keyword">if</span> @tokens.length &gt; <span class="hljs-number">0</span> <span class="hljs-keyword">then</span> @tokens[@tokens.length - <span class="hljs-number">1</span>][<span class="hljs-number">0</span>] <span class="hljs-keyword">else</span> <span class="hljs-string">&#x27;&#x27;</span>
    <span class="hljs-keyword">if</span> firstChar <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;&lt;&#x27;</span>
      match = JSX_IDENTIFIER.exec(@chunk[<span class="hljs-number">1.</span>..]) <span class="hljs-keyword">or</span> JSX_FRAGMENT_IDENTIFIER.exec(@chunk[<span class="hljs-number">1.</span>..])
      <span class="hljs-keyword">return</span> <span class="hljs-number">0</span> <span class="hljs-keyword">unless</span> match <span class="hljs-keyword">and</span> (
        @jsxDepth &gt; <span class="hljs-number">0</span> <span class="hljs-keyword">or</span></pre></div></div>
            
        </li>
        
        
        <li id="section-38">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-38">&#x00a7;</a>
              </div>
              <p>Not the right hand side of an unspaced comparison (i.e. <code>a&lt;b</code>).</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>        <span class="hljs-keyword">not</span> (prev = @prev()) <span class="hljs-keyword">or</span>
        prev.spaced <span class="hljs-keyword">or</span>
        prev[<span class="hljs-number">0</span>] <span class="hljs-keyword">not</span> <span class="hljs-keyword">in</span> COMPARABLE_LEFT_SIDE
      )
      [input, id] = match
      fullId = id
      <span class="hljs-keyword">if</span> <span class="hljs-string">&#x27;.&#x27;</span> <span class="hljs-keyword">in</span> id
        [id, properties...] = id.split <span class="hljs-string">&#x27;.&#x27;</span>
      <span class="hljs-keyword">else</span>
        properties = []
      tagToken = @token <span class="hljs-string">&#x27;JSX_TAG&#x27;</span>, id,
        length: id.length + <span class="hljs-number">1</span>
        data:
          openingBracketToken: @makeToken <span class="hljs-string">&#x27;&lt;&#x27;</span>, <span class="hljs-string">&#x27;&lt;&#x27;</span>
          tagNameToken: @makeToken <span class="hljs-string">&#x27;IDENTIFIER&#x27;</span>, id, offset: <span class="hljs-number">1</span>
      offset = id.length + <span class="hljs-number">1</span>
      <span class="hljs-keyword">for</span> property <span class="hljs-keyword">in</span> properties
        @token <span class="hljs-string">&#x27;.&#x27;</span>, <span class="hljs-string">&#x27;.&#x27;</span>, {offset}
        offset += <span class="hljs-number">1</span>
        @token <span class="hljs-string">&#x27;PROPERTY&#x27;</span>, property, {offset}
        offset += property.length
      @token <span class="hljs-string">&#x27;CALL_START&#x27;</span>, <span class="hljs-string">&#x27;(&#x27;</span>, generated: <span class="hljs-literal">yes</span>
      @token <span class="hljs-string">&#x27;[&#x27;</span>, <span class="hljs-string">&#x27;[&#x27;</span>, generated: <span class="hljs-literal">yes</span>
      @ends.push {tag: <span class="hljs-string">&#x27;/&gt;&#x27;</span>, origin: tagToken, name: id, properties}
      @jsxDepth++
      <span class="hljs-keyword">return</span> fullId.length + <span class="hljs-number">1</span>
    <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> jsxTag = @atJSXTag()
      <span class="hljs-keyword">if</span> @chunk[..<span class="hljs-number">.2</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;/&gt;&#x27;</span> <span class="hljs-comment"># Self-closing tag.</span>
        @pair <span class="hljs-string">&#x27;/&gt;&#x27;</span>
        @token <span class="hljs-string">&#x27;]&#x27;</span>, <span class="hljs-string">&#x27;]&#x27;</span>,
          length: <span class="hljs-number">2</span>
          generated: <span class="hljs-literal">yes</span>
        @token <span class="hljs-string">&#x27;CALL_END&#x27;</span>, <span class="hljs-string">&#x27;)&#x27;</span>,
          length: <span class="hljs-number">2</span>
          generated: <span class="hljs-literal">yes</span>
          data:
            selfClosingSlashToken: @makeToken <span class="hljs-string">&#x27;/&#x27;</span>, <span class="hljs-string">&#x27;/&#x27;</span>
            closingBracketToken: @makeToken <span class="hljs-string">&#x27;&gt;&#x27;</span>, <span class="hljs-string">&#x27;&gt;&#x27;</span>, offset: <span class="hljs-number">1</span>
        @jsxDepth--
        <span class="hljs-keyword">return</span> <span class="hljs-number">2</span>
      <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> firstChar <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;{&#x27;</span>
        <span class="hljs-keyword">if</span> prevChar <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;:&#x27;</span></pre></div></div>
            
        </li>
        
        
        <li id="section-39">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-39">&#x00a7;</a>
              </div>
              <p>This token represents the start of a JSX attribute value
that’s an expression (e.g. the <code>{b}</code> in <code>&lt;div a={b} /&gt;</code>).
Our grammar represents the beginnings of expressions as <code>(</code>
tokens, so make this into a <code>(</code> token that displays as <code>{</code>.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>          token = @token <span class="hljs-string">&#x27;(&#x27;</span>, <span class="hljs-string">&#x27;{&#x27;</span>
          @jsxObjAttribute[@jsxDepth] = <span class="hljs-literal">no</span></pre></div></div>
            
        </li>
        
        
        <li id="section-40">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-40">&#x00a7;</a>
              </div>
              <p>tag attribute name as JSX</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>          addTokenData @tokens[@tokens.length - <span class="hljs-number">3</span>],
            jsx: <span class="hljs-literal">yes</span>
        <span class="hljs-keyword">else</span>
          token = @token <span class="hljs-string">&#x27;{&#x27;</span>, <span class="hljs-string">&#x27;{&#x27;</span>
          @jsxObjAttribute[@jsxDepth] = <span class="hljs-literal">yes</span>
        @ends.push {tag: <span class="hljs-string">&#x27;}&#x27;</span>, origin: token}
        <span class="hljs-keyword">return</span> <span class="hljs-number">1</span>
      <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> firstChar <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;&gt;&#x27;</span> <span class="hljs-comment"># end of opening tag</span></pre></div></div>
            
        </li>
        
        
        <li id="section-41">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-41">&#x00a7;</a>
              </div>
              <p>Ignore terminators inside a tag.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>        {origin: openingTagToken} = @pair <span class="hljs-string">&#x27;/&gt;&#x27;</span> <span class="hljs-comment"># As if the current tag was self-closing.</span>
        @token <span class="hljs-string">&#x27;]&#x27;</span>, <span class="hljs-string">&#x27;]&#x27;</span>,
          generated: <span class="hljs-literal">yes</span>
          data:
            closingBracketToken: @makeToken <span class="hljs-string">&#x27;&gt;&#x27;</span>, <span class="hljs-string">&#x27;&gt;&#x27;</span>
        @token <span class="hljs-string">&#x27;,&#x27;</span>, <span class="hljs-string">&#x27;JSX_COMMA&#x27;</span>, generated: <span class="hljs-literal">yes</span>
        {tokens, index: end} =
          @matchWithInterpolations INSIDE_JSX, <span class="hljs-string">&#x27;&gt;&#x27;</span>, <span class="hljs-string">&#x27;&lt;/&#x27;</span>, JSX_INTERPOLATION
        @mergeInterpolationTokens tokens, {endOffset: end, jsx: <span class="hljs-literal">yes</span>}, <span class="hljs-function"><span class="hljs-params">(value)</span> =&gt;</span>
          @validateUnicodeCodePointEscapes value, delimiter: <span class="hljs-string">&#x27;&gt;&#x27;</span>
        match = JSX_IDENTIFIER.exec(@chunk[end...]) <span class="hljs-keyword">or</span> JSX_FRAGMENT_IDENTIFIER.exec(@chunk[end...])
        <span class="hljs-keyword">if</span> <span class="hljs-keyword">not</span> match <span class="hljs-keyword">or</span> match[<span class="hljs-number">1</span>] <span class="hljs-keyword">isnt</span> <span class="hljs-string">&quot;<span class="hljs-subst">#{jsxTag.name}</span><span class="hljs-subst">#{(<span class="hljs-string">&quot;.<span class="hljs-subst">#{property}</span>&quot;</span> <span class="hljs-keyword">for</span> property <span class="hljs-keyword">in</span> jsxTag.properties).join <span class="hljs-string">&#x27;&#x27;</span>}</span>&quot;</span>
          @error <span class="hljs-string">&quot;expected corresponding JSX closing tag for <span class="hljs-subst">#{jsxTag.name}</span>&quot;</span>,
            jsxTag.origin.data.tagNameToken[<span class="hljs-number">2</span>]
        [, fullTagName] = match
        afterTag = end + fullTagName.length
        <span class="hljs-keyword">if</span> @chunk[afterTag] <span class="hljs-keyword">isnt</span> <span class="hljs-string">&#x27;&gt;&#x27;</span>
          @error <span class="hljs-string">&quot;missing closing &gt; after tag name&quot;</span>, offset: afterTag, length: <span class="hljs-number">1</span></pre></div></div>
            
        </li>
        
        
        <li id="section-42">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-42">&#x00a7;</a>
              </div>
              <p>-2/+2 for the opening <code>&lt;/</code> and +1 for the closing <code>&gt;</code>.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>        endToken = @token <span class="hljs-string">&#x27;CALL_END&#x27;</span>, <span class="hljs-string">&#x27;)&#x27;</span>,
          offset: end - <span class="hljs-number">2</span>
          length: fullTagName.length + <span class="hljs-number">3</span>
          generated: <span class="hljs-literal">yes</span>
          data:
            closingTagOpeningBracketToken: @makeToken <span class="hljs-string">&#x27;&lt;&#x27;</span>, <span class="hljs-string">&#x27;&lt;&#x27;</span>, offset: end - <span class="hljs-number">2</span>
            closingTagSlashToken: @makeToken <span class="hljs-string">&#x27;/&#x27;</span>, <span class="hljs-string">&#x27;/&#x27;</span>, offset: end - <span class="hljs-number">1</span></pre></div></div>
            
        </li>
        
        
        <li id="section-43">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-43">&#x00a7;</a>
              </div>
              <p>TODO: individual tokens for complex tag name? eg &lt; / A . B &gt;</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>            closingTagNameToken: @makeToken <span class="hljs-string">&#x27;IDENTIFIER&#x27;</span>, fullTagName, offset: end
            closingTagClosingBracketToken: @makeToken <span class="hljs-string">&#x27;&gt;&#x27;</span>, <span class="hljs-string">&#x27;&gt;&#x27;</span>, offset: end + fullTagName.length</pre></div></div>
            
        </li>
        
        
        <li id="section-44">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-44">&#x00a7;</a>
              </div>
              <p>make the closing tag location data more easily accessible to the grammar</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>        addTokenData openingTagToken, endToken.data
        @jsxDepth--
        <span class="hljs-keyword">return</span> afterTag + <span class="hljs-number">1</span>
      <span class="hljs-keyword">else</span>
        <span class="hljs-keyword">return</span> <span class="hljs-number">0</span>
    <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> @atJSXTag <span class="hljs-number">1</span>
      <span class="hljs-keyword">if</span> firstChar <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;}&#x27;</span>
        @pair firstChar
        <span class="hljs-keyword">if</span> @jsxObjAttribute[@jsxDepth]
          @token <span class="hljs-string">&#x27;}&#x27;</span>, <span class="hljs-string">&#x27;}&#x27;</span>
          @jsxObjAttribute[@jsxDepth] = <span class="hljs-literal">no</span>
        <span class="hljs-keyword">else</span>
          @token <span class="hljs-string">&#x27;)&#x27;</span>, <span class="hljs-string">&#x27;}&#x27;</span>
        @token <span class="hljs-string">&#x27;,&#x27;</span>, <span class="hljs-string">&#x27;,&#x27;</span>, generated: <span class="hljs-literal">yes</span>
        <span class="hljs-keyword">return</span> <span class="hljs-number">1</span>
      <span class="hljs-keyword">else</span>
        <span class="hljs-keyword">return</span> <span class="hljs-number">0</span>
    <span class="hljs-keyword">else</span>
      <span class="hljs-keyword">return</span> <span class="hljs-number">0</span>

  atJSXTag: <span class="hljs-function"><span class="hljs-params">(depth = <span class="hljs-number">0</span>)</span> -&gt;</span>
    <span class="hljs-keyword">return</span> <span class="hljs-literal">no</span> <span class="hljs-keyword">if</span> @jsxDepth <span class="hljs-keyword">is</span> <span class="hljs-number">0</span>
    i = @ends.length - <span class="hljs-number">1</span>
    i-- <span class="hljs-keyword">while</span> @ends[i]?.tag <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;OUTDENT&#x27;</span> <span class="hljs-keyword">or</span> depth-- &gt; <span class="hljs-number">0</span> <span class="hljs-comment"># Ignore indents.</span>
    last = @ends[i]
    last?.tag <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;/&gt;&#x27;</span> <span class="hljs-keyword">and</span> last</pre></div></div>
            
        </li>
        
        
        <li id="section-45">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-45">&#x00a7;</a>
              </div>
              <p>We treat all other single characters as a token. E.g.: <code>( ) , . !</code>
Multi-character operators are also literal tokens, so that Jison can assign
the proper order of operations. There are some symbols that we tag specially
here. <code>;</code> and newlines are both treated as a <code>TERMINATOR</code>, we distinguish
parentheses that indicate a method call from regular parentheses, and so on.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>  literalToken: <span class="hljs-function">-&gt;</span>
    <span class="hljs-keyword">if</span> match = OPERATOR.exec @chunk
      [value] = match
      @tagParameters() <span class="hljs-keyword">if</span> CODE.test value
    <span class="hljs-keyword">else</span>
      value = @chunk.charAt <span class="hljs-number">0</span>
    tag  = value
    prev = @prev()

    <span class="hljs-keyword">if</span> prev <span class="hljs-keyword">and</span> value <span class="hljs-keyword">in</span> [<span class="hljs-string">&#x27;=&#x27;</span>, COMPOUND_ASSIGN...]
      skipToken = <span class="hljs-literal">false</span>
      <span class="hljs-keyword">if</span> value <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;=&#x27;</span> <span class="hljs-keyword">and</span> prev[<span class="hljs-number">1</span>] <span class="hljs-keyword">in</span> [<span class="hljs-string">&#x27;||&#x27;</span>, <span class="hljs-string">&#x27;&amp;&amp;&#x27;</span>] <span class="hljs-keyword">and</span> <span class="hljs-keyword">not</span> prev.spaced
        prev[<span class="hljs-number">0</span>] = <span class="hljs-string">&#x27;COMPOUND_ASSIGN&#x27;</span>
        prev[<span class="hljs-number">1</span>] += <span class="hljs-string">&#x27;=&#x27;</span>
        prev.data.original += <span class="hljs-string">&#x27;=&#x27;</span> <span class="hljs-keyword">if</span> prev.data?.original
        prev[<span class="hljs-number">2</span>].range = [
          prev[<span class="hljs-number">2</span>].range[<span class="hljs-number">0</span>]
          prev[<span class="hljs-number">2</span>].range[<span class="hljs-number">1</span>] + <span class="hljs-number">1</span>
        ]
        prev[<span class="hljs-number">2</span>].last_column += <span class="hljs-number">1</span>
        prev[<span class="hljs-number">2</span>].last_column_exclusive += <span class="hljs-number">1</span>
        prev = @tokens[@tokens.length - <span class="hljs-number">2</span>]
        skipToken = <span class="hljs-literal">true</span>
      <span class="hljs-keyword">if</span> prev <span class="hljs-keyword">and</span> prev[<span class="hljs-number">0</span>] <span class="hljs-keyword">isnt</span> <span class="hljs-string">&#x27;PROPERTY&#x27;</span>
        origin = prev.origin ? prev
        message = isUnassignable prev[<span class="hljs-number">1</span>], origin[<span class="hljs-number">1</span>]
        @error message, origin[<span class="hljs-number">2</span>] <span class="hljs-keyword">if</span> message
      <span class="hljs-keyword">return</span> value.length <span class="hljs-keyword">if</span> skipToken

    <span class="hljs-keyword">if</span> value <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;(&#x27;</span> <span class="hljs-keyword">and</span> prev?[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;IMPORT&#x27;</span>
      prev[<span class="hljs-number">0</span>] = <span class="hljs-string">&#x27;DYNAMIC_IMPORT&#x27;</span>

    <span class="hljs-keyword">if</span> value <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;{&#x27;</span> <span class="hljs-keyword">and</span> @seenImport
      @importSpecifierList = <span class="hljs-literal">yes</span>
    <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> @importSpecifierList <span class="hljs-keyword">and</span> value <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;}&#x27;</span>
      @importSpecifierList = <span class="hljs-literal">no</span>
    <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> value <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;{&#x27;</span> <span class="hljs-keyword">and</span> prev?[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;EXPORT&#x27;</span>
      @exportSpecifierList = <span class="hljs-literal">yes</span>
    <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> @exportSpecifierList <span class="hljs-keyword">and</span> value <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;}&#x27;</span>
      @exportSpecifierList = <span class="hljs-literal">no</span>

    <span class="hljs-keyword">if</span> value <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;;&#x27;</span>
      @error <span class="hljs-string">&#x27;unexpected ;&#x27;</span> <span class="hljs-keyword">if</span> prev?[<span class="hljs-number">0</span>] <span class="hljs-keyword">in</span> [<span class="hljs-string">&#x27;=&#x27;</span>, UNFINISHED...]
      @seenFor = @seenImport = @seenExport = <span class="hljs-literal">no</span>
      tag = <span class="hljs-string">&#x27;TERMINATOR&#x27;</span>
    <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> value <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;*&#x27;</span> <span class="hljs-keyword">and</span> prev?[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;EXPORT&#x27;</span>
      tag = <span class="hljs-string">&#x27;EXPORT_ALL&#x27;</span>
    <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> value <span class="hljs-keyword">in</span> MATH            <span class="hljs-keyword">then</span> tag = <span class="hljs-string">&#x27;MATH&#x27;</span>
    <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> value <span class="hljs-keyword">in</span> COMPARE         <span class="hljs-keyword">then</span> tag = <span class="hljs-string">&#x27;COMPARE&#x27;</span>
    <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> value <span class="hljs-keyword">in</span> COMPOUND_ASSIGN <span class="hljs-keyword">then</span> tag = <span class="hljs-string">&#x27;COMPOUND_ASSIGN&#x27;</span>
    <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> value <span class="hljs-keyword">in</span> UNARY           <span class="hljs-keyword">then</span> tag = <span class="hljs-string">&#x27;UNARY&#x27;</span>
    <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> value <span class="hljs-keyword">in</span> UNARY_MATH      <span class="hljs-keyword">then</span> tag = <span class="hljs-string">&#x27;UNARY_MATH&#x27;</span>
    <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> value <span class="hljs-keyword">in</span> SHIFT           <span class="hljs-keyword">then</span> tag = <span class="hljs-string">&#x27;SHIFT&#x27;</span>
    <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> value <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;?&#x27;</span> <span class="hljs-keyword">and</span> prev?.spaced <span class="hljs-keyword">then</span> tag = <span class="hljs-string">&#x27;BIN?&#x27;</span>
    <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> prev
      <span class="hljs-keyword">if</span> value <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;(&#x27;</span> <span class="hljs-keyword">and</span> <span class="hljs-keyword">not</span> prev.spaced <span class="hljs-keyword">and</span> prev[<span class="hljs-number">0</span>] <span class="hljs-keyword">in</span> CALLABLE
        prev[<span class="hljs-number">0</span>] = <span class="hljs-string">&#x27;FUNC_EXIST&#x27;</span> <span class="hljs-keyword">if</span> prev[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;?&#x27;</span>
        tag = <span class="hljs-string">&#x27;CALL_START&#x27;</span>
      <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> value <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;[&#x27;</span> <span class="hljs-keyword">and</span> ((prev[<span class="hljs-number">0</span>] <span class="hljs-keyword">in</span> INDEXABLE <span class="hljs-keyword">and</span> <span class="hljs-keyword">not</span> prev.spaced) <span class="hljs-keyword">or</span>
         (prev[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;::&#x27;</span>)) <span class="hljs-comment"># `.prototype` can’t be a method you can call.</span>
        tag = <span class="hljs-string">&#x27;INDEX_START&#x27;</span>
        <span class="hljs-keyword">switch</span> prev[<span class="hljs-number">0</span>]
          <span class="hljs-keyword">when</span> <span class="hljs-string">&#x27;?&#x27;</span>  <span class="hljs-keyword">then</span> prev[<span class="hljs-number">0</span>] = <span class="hljs-string">&#x27;INDEX_SOAK&#x27;</span>
    token = @makeToken tag, value
    <span class="hljs-keyword">switch</span> value
      <span class="hljs-keyword">when</span> <span class="hljs-string">&#x27;(&#x27;</span>, <span class="hljs-string">&#x27;{&#x27;</span>, <span class="hljs-string">&#x27;[&#x27;</span> <span class="hljs-keyword">then</span> @ends.push {tag: INVERSES[value], origin: token}
      <span class="hljs-keyword">when</span> <span class="hljs-string">&#x27;)&#x27;</span>, <span class="hljs-string">&#x27;}&#x27;</span>, <span class="hljs-string">&#x27;]&#x27;</span> <span class="hljs-keyword">then</span> @pair value
    @tokens.push @makeToken tag, value
    value.length</pre></div></div>
            
        </li>
        
        
        <li id="section-46">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-46">&#x00a7;</a>
              </div>
              <h2 id="token-manipulators">Token Manipulators</h2>

            </div>
            
        </li>
        
        
        <li id="section-47">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-47">&#x00a7;</a>
              </div>
              
            </div>
            
        </li>
        
        
        <li id="section-48">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-48">&#x00a7;</a>
              </div>
              <p>A source of ambiguity in our grammar used to be parameter lists in function
definitions versus argument lists in function calls. Walk backwards, tagging
parameters specially in order to make things easier for the parser.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>  tagParameters: <span class="hljs-function">-&gt;</span>
    <span class="hljs-keyword">return</span> @tagDoIife() <span class="hljs-keyword">if</span> @tag() <span class="hljs-keyword">isnt</span> <span class="hljs-string">&#x27;)&#x27;</span>
    stack = []
    {tokens} = this
    i = tokens.length
    paramEndToken = tokens[--i]
    paramEndToken[<span class="hljs-number">0</span>] = <span class="hljs-string">&#x27;PARAM_END&#x27;</span>
    <span class="hljs-keyword">while</span> tok = tokens[--i]
      <span class="hljs-keyword">switch</span> tok[<span class="hljs-number">0</span>]
        <span class="hljs-keyword">when</span> <span class="hljs-string">&#x27;)&#x27;</span>
          stack.push tok
        <span class="hljs-keyword">when</span> <span class="hljs-string">&#x27;(&#x27;</span>, <span class="hljs-string">&#x27;CALL_START&#x27;</span>
          <span class="hljs-keyword">if</span> stack.length <span class="hljs-keyword">then</span> stack.pop()
          <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> tok[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;(&#x27;</span>
            tok[<span class="hljs-number">0</span>] = <span class="hljs-string">&#x27;PARAM_START&#x27;</span>
            <span class="hljs-keyword">return</span> @tagDoIife i - <span class="hljs-number">1</span>
          <span class="hljs-keyword">else</span>
            paramEndToken[<span class="hljs-number">0</span>] = <span class="hljs-string">&#x27;CALL_END&#x27;</span>
            <span class="hljs-keyword">return</span> this
    this</pre></div></div>
            
        </li>
        
        
        <li id="section-49">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-49">&#x00a7;</a>
              </div>
              <p>Tag <code>do</code> followed by a function differently than <code>do</code> followed by eg an
identifier to allow for different grammar precedence</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>  tagDoIife: <span class="hljs-function"><span class="hljs-params">(tokenIndex)</span> -&gt;</span>
    tok = @tokens[tokenIndex ? @tokens.length - <span class="hljs-number">1</span>]
    <span class="hljs-keyword">return</span> this <span class="hljs-keyword">unless</span> tok?[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;DO&#x27;</span>
    tok[<span class="hljs-number">0</span>] = <span class="hljs-string">&#x27;DO_IIFE&#x27;</span>
    this</pre></div></div>
            
        </li>
        
        
        <li id="section-50">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-50">&#x00a7;</a>
              </div>
              <p>Close up all remaining open blocks at the end of the file.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>  closeIndentation: <span class="hljs-function">-&gt;</span>
    @outdentToken moveOut: @indent, indentSize: <span class="hljs-number">0</span></pre></div></div>
            
        </li>
        
        
        <li id="section-51">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-51">&#x00a7;</a>
              </div>
              <p>Match the contents of a delimited token and expand variables and expressions
inside it using Ruby-like notation for substitution of arbitrary
expressions.</p>
<pre><code><span class="hljs-string">&quot;Hello <span class="hljs-subst">#{name.capitalize()}</span>.&quot;</span>
</code></pre>
<p>If it encounters an interpolation, this method will recursively create a new
Lexer and tokenize until the <code>{</code> of <code>#{</code> is balanced with a <code>}</code>.</p>
<ul>
<li><code>regex</code> matches the contents of a token (but not <code>delimiter</code>, and not
<code>#{</code> if interpolations are desired).</li>
<li><code>delimiter</code> is the delimiter of the token. Examples are <code>&#39;</code>, <code>&quot;</code>, <code>&#39;&#39;&#39;</code>,
<code>&quot;&quot;&quot;</code> and <code>///</code>.</li>
<li><code>closingDelimiter</code> is different from <code>delimiter</code> only in JSX</li>
<li><code>interpolators</code> matches the start of an interpolation, for JSX it’s both
<code>{</code> and <code>&lt;</code> (i.e. nested JSX tag)</li>
</ul>
<p>This method allows us to have strings within interpolations within strings,
ad infinitum.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>  matchWithInterpolations: <span class="hljs-function"><span class="hljs-params">(regex, delimiter, closingDelimiter = delimiter, interpolators = <span class="hljs-regexp">/^#\{/</span>)</span> -&gt;</span>
    tokens = []
    offsetInChunk = delimiter.length
    <span class="hljs-keyword">return</span> <span class="hljs-literal">null</span> <span class="hljs-keyword">unless</span> @chunk[...offsetInChunk] <span class="hljs-keyword">is</span> delimiter
    str = @chunk[offsetInChunk..]
    <span class="hljs-keyword">loop</span>
      [strPart] = regex.exec str

      @validateEscapes strPart, {isRegex: delimiter.charAt(<span class="hljs-number">0</span>) <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;/&#x27;</span>, offsetInChunk}</pre></div></div>
            
        </li>
        
        
        <li id="section-52">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-52">&#x00a7;</a>
              </div>
              <p>Push a fake <code>&#39;NEOSTRING&#39;</code> token, which will get turned into a real string later.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>      tokens.push @makeToken <span class="hljs-string">&#x27;NEOSTRING&#x27;</span>, strPart, offset: offsetInChunk

      str = str[strPart.length..]
      offsetInChunk += strPart.length

      <span class="hljs-keyword">break</span> <span class="hljs-keyword">unless</span> match = interpolators.exec str
      [interpolator] = match</pre></div></div>
            
        </li>
        
        
        <li id="section-53">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-53">&#x00a7;</a>
              </div>
              <p>To remove the <code>#</code> in <code>#{</code>.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>      interpolationOffset = interpolator.length - <span class="hljs-number">1</span>
      [line, column, offset] = @getLineAndColumnFromChunk offsetInChunk + interpolationOffset
      rest = str[interpolationOffset..]
      {tokens: nested, index} =
        <span class="hljs-keyword">new</span> Lexer().tokenize rest, {line, column, offset, untilBalanced: <span class="hljs-literal">on</span>, @locationDataCompensations}</pre></div></div>
            
        </li>
        
        
        <li id="section-54">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-54">&#x00a7;</a>
              </div>
              <p>Account for the <code>#</code> in <code>#{</code>.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>      index += interpolationOffset

      braceInterpolator = str[index - <span class="hljs-number">1</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;}&#x27;</span>
      <span class="hljs-keyword">if</span> braceInterpolator</pre></div></div>
            
        </li>
        
        
        <li id="section-55">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-55">&#x00a7;</a>
              </div>
              <p>Turn the leading and trailing <code>{</code> and <code>}</code> into parentheses. Unnecessary
parentheses will be removed later.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>        [open, ..., close] = nested
        open[<span class="hljs-number">0</span>]  = <span class="hljs-string">&#x27;INTERPOLATION_START&#x27;</span>
        open[<span class="hljs-number">1</span>]  = <span class="hljs-string">&#x27;(&#x27;</span>
        open[<span class="hljs-number">2</span>].first_column -= interpolationOffset
        open[<span class="hljs-number">2</span>].range = [
          open[<span class="hljs-number">2</span>].range[<span class="hljs-number">0</span>] - interpolationOffset
          open[<span class="hljs-number">2</span>].range[<span class="hljs-number">1</span>]
        ]
        close[<span class="hljs-number">0</span>]  = <span class="hljs-string">&#x27;INTERPOLATION_END&#x27;</span>
        close[<span class="hljs-number">1</span>] = <span class="hljs-string">&#x27;)&#x27;</span>
        close.origin = [<span class="hljs-string">&#x27;&#x27;</span>, <span class="hljs-string">&#x27;end of interpolation&#x27;</span>, close[<span class="hljs-number">2</span>]]</pre></div></div>
            
        </li>
        
        
        <li id="section-56">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-56">&#x00a7;</a>
              </div>
              <p>Remove leading <code>&#39;TERMINATOR&#39;</code> (if any).</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>      nested.splice <span class="hljs-number">1</span>, <span class="hljs-number">1</span> <span class="hljs-keyword">if</span> nested[<span class="hljs-number">1</span>]?[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;TERMINATOR&#x27;</span></pre></div></div>
            
        </li>
        
        
        <li id="section-57">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-57">&#x00a7;</a>
              </div>
              <p>Remove trailing <code>&#39;INDENT&#39;/&#39;OUTDENT&#39;</code> pair (if any).</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>      nested.splice <span class="hljs-number">-3</span>, <span class="hljs-number">2</span> <span class="hljs-keyword">if</span> nested[nested.length - <span class="hljs-number">3</span>]?[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;INDENT&#x27;</span> <span class="hljs-keyword">and</span> nested[nested.length - <span class="hljs-number">2</span>][<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;OUTDENT&#x27;</span>

      <span class="hljs-keyword">unless</span> braceInterpolator</pre></div></div>
            
        </li>
        
        
        <li id="section-58">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-58">&#x00a7;</a>
              </div>
              <p>We are not using <code>{</code> and <code>}</code>, so wrap the interpolated tokens instead.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>        open = @makeToken <span class="hljs-string">&#x27;INTERPOLATION_START&#x27;</span>, <span class="hljs-string">&#x27;(&#x27;</span>, offset: offsetInChunk,         length: <span class="hljs-number">0</span>, generated: <span class="hljs-literal">yes</span>
        close = @makeToken <span class="hljs-string">&#x27;INTERPOLATION_END&#x27;</span>, <span class="hljs-string">&#x27;)&#x27;</span>,  offset: offsetInChunk + index, length: <span class="hljs-number">0</span>, generated: <span class="hljs-literal">yes</span>
        nested = [open, nested..., close]</pre></div></div>
            
        </li>
        
        
        <li id="section-59">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-59">&#x00a7;</a>
              </div>
              <p>Push a fake <code>&#39;TOKENS&#39;</code> token, which will get turned into real tokens later.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>      tokens.push [<span class="hljs-string">&#x27;TOKENS&#x27;</span>, nested]

      str = str[index..]
      offsetInChunk += index

    <span class="hljs-keyword">unless</span> str[...closingDelimiter.length] <span class="hljs-keyword">is</span> closingDelimiter
      @error <span class="hljs-string">&quot;missing <span class="hljs-subst">#{closingDelimiter}</span>&quot;</span>, length: delimiter.length

    {tokens, index: offsetInChunk + closingDelimiter.length}</pre></div></div>
            
        </li>
        
        
        <li id="section-60">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-60">&#x00a7;</a>
              </div>
              <p>Merge the array <code>tokens</code> of the fake token types <code>&#39;TOKENS&#39;</code> and <code>&#39;NEOSTRING&#39;</code>
(as returned by <code>matchWithInterpolations</code>) into the token stream. The value
of <code>&#39;NEOSTRING&#39;</code>s are converted using <code>fn</code> and turned into strings using
<code>options</code> first.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>  mergeInterpolationTokens: <span class="hljs-function"><span class="hljs-params">(tokens, options, fn)</span> -&gt;</span>
    {quote, indent, double, heregex, endOffset, jsx} = options

    <span class="hljs-keyword">if</span> tokens.length &gt; <span class="hljs-number">1</span>
      lparen = @token <span class="hljs-string">&#x27;STRING_START&#x27;</span>, <span class="hljs-string">&#x27;(&#x27;</span>, length: quote?.length ? <span class="hljs-number">0</span>, data: {quote}, generated: <span class="hljs-keyword">not</span> quote?.length

    firstIndex = @tokens.length
    $ = tokens.length - <span class="hljs-number">1</span>
    <span class="hljs-keyword">for</span> token, i <span class="hljs-keyword">in</span> tokens
      [tag, value] = token
      <span class="hljs-keyword">switch</span> tag
        <span class="hljs-keyword">when</span> <span class="hljs-string">&#x27;TOKENS&#x27;</span></pre></div></div>
            
        </li>
        
        
        <li id="section-61">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-61">&#x00a7;</a>
              </div>
              <p>There are comments (and nothing else) in this interpolation.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>          <span class="hljs-keyword">if</span> value.length <span class="hljs-keyword">is</span> <span class="hljs-number">2</span> <span class="hljs-keyword">and</span> (value[<span class="hljs-number">0</span>].comments <span class="hljs-keyword">or</span> value[<span class="hljs-number">1</span>].comments)
            placeholderToken = @makeToken <span class="hljs-string">&#x27;JS&#x27;</span>, <span class="hljs-string">&#x27;&#x27;</span>, generated: <span class="hljs-literal">yes</span></pre></div></div>
            
        </li>
        
        
        <li id="section-62">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-62">&#x00a7;</a>
              </div>
              <p>Use the same location data as the first parenthesis.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>            placeholderToken[<span class="hljs-number">2</span>] = value[<span class="hljs-number">0</span>][<span class="hljs-number">2</span>]
            <span class="hljs-keyword">for</span> val <span class="hljs-keyword">in</span> value <span class="hljs-keyword">when</span> val.comments
              placeholderToken.comments ?= []
              placeholderToken.comments.push val.comments...
            value.splice <span class="hljs-number">1</span>, <span class="hljs-number">0</span>, placeholderToken</pre></div></div>
            
        </li>
        
        
        <li id="section-63">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-63">&#x00a7;</a>
              </div>
              <p>Push all the tokens in the fake <code>&#39;TOKENS&#39;</code> token. These already have
sane location data.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>          locationToken = value[<span class="hljs-number">0</span>]
          tokensToPush = value
        <span class="hljs-keyword">when</span> <span class="hljs-string">&#x27;NEOSTRING&#x27;</span></pre></div></div>
            
        </li>
        
        
        <li id="section-64">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-64">&#x00a7;</a>
              </div>
              <p>Convert <code>&#39;NEOSTRING&#39;</code> into <code>&#39;STRING&#39;</code>.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>          converted = fn.call this, token[<span class="hljs-number">1</span>], i
          addTokenData token, initialChunk: <span class="hljs-literal">yes</span> <span class="hljs-keyword">if</span> i <span class="hljs-keyword">is</span> <span class="hljs-number">0</span>
          addTokenData token, finalChunk: <span class="hljs-literal">yes</span>   <span class="hljs-keyword">if</span> i <span class="hljs-keyword">is</span> $
          addTokenData token, {indent, quote, double}
          addTokenData token, {heregex} <span class="hljs-keyword">if</span> heregex
          addTokenData token, {jsx} <span class="hljs-keyword">if</span> jsx
          token[<span class="hljs-number">0</span>] = <span class="hljs-string">&#x27;STRING&#x27;</span>
          token[<span class="hljs-number">1</span>] = <span class="hljs-string">&#x27;&quot;&#x27;</span> + converted + <span class="hljs-string">&#x27;&quot;&#x27;</span>
          <span class="hljs-keyword">if</span> tokens.length <span class="hljs-keyword">is</span> <span class="hljs-number">1</span> <span class="hljs-keyword">and</span> quote?
            token[<span class="hljs-number">2</span>].first_column -= quote.length
            <span class="hljs-keyword">if</span> token[<span class="hljs-number">1</span>].substr(<span class="hljs-number">-2</span>, <span class="hljs-number">1</span>) <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;\n&#x27;</span>
              token[<span class="hljs-number">2</span>].last_line += <span class="hljs-number">1</span>
              token[<span class="hljs-number">2</span>].last_column = quote.length - <span class="hljs-number">1</span>
            <span class="hljs-keyword">else</span>
              token[<span class="hljs-number">2</span>].last_column += quote.length
              token[<span class="hljs-number">2</span>].last_column -= <span class="hljs-number">1</span> <span class="hljs-keyword">if</span> token[<span class="hljs-number">1</span>].length <span class="hljs-keyword">is</span> <span class="hljs-number">2</span>
            token[<span class="hljs-number">2</span>].last_column_exclusive += quote.length
            token[<span class="hljs-number">2</span>].range = [
              token[<span class="hljs-number">2</span>].range[<span class="hljs-number">0</span>] - quote.length
              token[<span class="hljs-number">2</span>].range[<span class="hljs-number">1</span>] + quote.length
            ]
          locationToken = token
          tokensToPush = [token]
      @tokens.push tokensToPush...

    <span class="hljs-keyword">if</span> lparen
      [..., lastToken] = tokens
      lparen.origin = [<span class="hljs-string">&#x27;STRING&#x27;</span>, <span class="hljs-literal">null</span>,
        first_line:            lparen[<span class="hljs-number">2</span>].first_line
        first_column:          lparen[<span class="hljs-number">2</span>].first_column
        last_line:             lastToken[<span class="hljs-number">2</span>].last_line
        last_column:           lastToken[<span class="hljs-number">2</span>].last_column
        last_line_exclusive:   lastToken[<span class="hljs-number">2</span>].last_line_exclusive
        last_column_exclusive: lastToken[<span class="hljs-number">2</span>].last_column_exclusive
        range: [
          lparen[<span class="hljs-number">2</span>].range[<span class="hljs-number">0</span>]
          lastToken[<span class="hljs-number">2</span>].range[<span class="hljs-number">1</span>]
        ]
      ]
      lparen[<span class="hljs-number">2</span>] = lparen.origin[<span class="hljs-number">2</span>] <span class="hljs-keyword">unless</span> quote?.length
      rparen = @token <span class="hljs-string">&#x27;STRING_END&#x27;</span>, <span class="hljs-string">&#x27;)&#x27;</span>, offset: endOffset - (quote ? <span class="hljs-string">&#x27;&#x27;</span>).length, length: quote?.length ? <span class="hljs-number">0</span>, generated: <span class="hljs-keyword">not</span> quote?.length</pre></div></div>
            
        </li>
        
        
        <li id="section-65">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-65">&#x00a7;</a>
              </div>
              <p>Pairs up a closing token, ensuring that all listed pairs of tokens are
correctly balanced throughout the course of the token stream.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>  pair: <span class="hljs-function"><span class="hljs-params">(tag)</span> -&gt;</span>
    [..., prev] = @ends
    <span class="hljs-keyword">unless</span> tag <span class="hljs-keyword">is</span> wanted = prev?.tag
      @error <span class="hljs-string">&quot;unmatched <span class="hljs-subst">#{tag}</span>&quot;</span> <span class="hljs-keyword">unless</span> <span class="hljs-string">&#x27;OUTDENT&#x27;</span> <span class="hljs-keyword">is</span> wanted</pre></div></div>
            
        </li>
        
        
        <li id="section-66">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-66">&#x00a7;</a>
              </div>
              <p>Auto-close <code>INDENT</code> to support syntax like this:</p>
<pre><code>el.click(<span class="hljs-function"><span class="hljs-params">(event)</span> -&gt;</span>
  el.hide())
</code></pre>

            </div>
            
            <div class="content"><div class='highlight'><pre>      [..., lastIndent] = @indents
      @outdentToken moveOut: lastIndent, noNewlines: <span class="hljs-literal">true</span>
      <span class="hljs-keyword">return</span> @pair tag
    @ends.pop()</pre></div></div>
            
        </li>
        
        
        <li id="section-67">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-67">&#x00a7;</a>
              </div>
              <h2 id="helpers">Helpers</h2>

            </div>
            
        </li>
        
        
        <li id="section-68">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-68">&#x00a7;</a>
              </div>
              
            </div>
            
        </li>
        
        
        <li id="section-69">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-69">&#x00a7;</a>
              </div>
              <p>Compensate for the things we strip out initially (e.g. carriage returns)
so that location data stays accurate with respect to the original source file.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>  getLocationDataCompensation: <span class="hljs-function"><span class="hljs-params">(start, end)</span> -&gt;</span>
    totalCompensation = <span class="hljs-number">0</span>
    initialEnd = end
    current = start
    <span class="hljs-keyword">while</span> current &lt;= end
      <span class="hljs-keyword">break</span> <span class="hljs-keyword">if</span> current <span class="hljs-keyword">is</span> end <span class="hljs-keyword">and</span> start <span class="hljs-keyword">isnt</span> initialEnd
      compensation = @locationDataCompensations[current]
      <span class="hljs-keyword">if</span> compensation?
        totalCompensation += compensation
        end += compensation
      current++
    <span class="hljs-keyword">return</span> totalCompensation</pre></div></div>
            
        </li>
        
        
        <li id="section-70">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-70">&#x00a7;</a>
              </div>
              <p>Returns the line and column number from an offset into the current chunk.</p>
<p><code>offset</code> is a number of characters into <code>@chunk</code>.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>  getLineAndColumnFromChunk: <span class="hljs-function"><span class="hljs-params">(offset)</span> -&gt;</span>
    compensation = @getLocationDataCompensation @chunkOffset, @chunkOffset + offset

    <span class="hljs-keyword">if</span> offset <span class="hljs-keyword">is</span> <span class="hljs-number">0</span>
      <span class="hljs-keyword">return</span> [@chunkLine, @chunkColumn + compensation, @chunkOffset + compensation]

    <span class="hljs-keyword">if</span> offset &gt;= @chunk.length
      string = @chunk
    <span class="hljs-keyword">else</span>
      string = @chunk[..offset<span class="hljs-number">-1</span>]

    lineCount = count string, <span class="hljs-string">&#x27;\n&#x27;</span>

    column = @chunkColumn
    <span class="hljs-keyword">if</span> lineCount &gt; <span class="hljs-number">0</span>
      [..., lastLine] = string.split <span class="hljs-string">&#x27;\n&#x27;</span>
      column = lastLine.length
      previousLinesCompensation = @getLocationDataCompensation @chunkOffset, @chunkOffset + offset - column</pre></div></div>
            
        </li>
        
        
        <li id="section-71">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-71">&#x00a7;</a>
              </div>
              <p>Don’t recompensate for initially inserted newline.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>      previousLinesCompensation = <span class="hljs-number">0</span> <span class="hljs-keyword">if</span> previousLinesCompensation &lt; <span class="hljs-number">0</span>
      columnCompensation = @getLocationDataCompensation(
        @chunkOffset + offset + previousLinesCompensation - column
        @chunkOffset + offset + previousLinesCompensation
      )
    <span class="hljs-keyword">else</span>
      column += string.length
      columnCompensation = compensation

    [@chunkLine + lineCount, column + columnCompensation, @chunkOffset + offset + compensation]

  makeLocationData: <span class="hljs-function"><span class="hljs-params">({ offsetInChunk, length })</span> -&gt;</span>
    locationData = range: []
    [locationData.first_line, locationData.first_column, locationData.range[<span class="hljs-number">0</span>]] =
      @getLineAndColumnFromChunk offsetInChunk</pre></div></div>
            
        </li>
        
        
        <li id="section-72">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-72">&#x00a7;</a>
              </div>
              <p>Use length - 1 for the final offset - we’re supplying the last_line and the last_column,
so if last_column == first_column, then we’re looking at a character of length 1.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>    lastCharacter = <span class="hljs-keyword">if</span> length &gt; <span class="hljs-number">0</span> <span class="hljs-keyword">then</span> (length - <span class="hljs-number">1</span>) <span class="hljs-keyword">else</span> <span class="hljs-number">0</span>
    [locationData.last_line, locationData.last_column, endOffset] =
      @getLineAndColumnFromChunk offsetInChunk + lastCharacter
    [locationData.last_line_exclusive, locationData.last_column_exclusive] =
      @getLineAndColumnFromChunk offsetInChunk + lastCharacter + (<span class="hljs-keyword">if</span> length &gt; <span class="hljs-number">0</span> <span class="hljs-keyword">then</span> <span class="hljs-number">1</span> <span class="hljs-keyword">else</span> <span class="hljs-number">0</span>)
    locationData.range[<span class="hljs-number">1</span>] = <span class="hljs-keyword">if</span> length &gt; <span class="hljs-number">0</span> <span class="hljs-keyword">then</span> endOffset + <span class="hljs-number">1</span> <span class="hljs-keyword">else</span> endOffset

    locationData</pre></div></div>
            
        </li>
        
        
        <li id="section-73">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-73">&#x00a7;</a>
              </div>
              <p>Same as <code>token</code>, except this just returns the token without adding it
to the results.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>  makeToken: <span class="hljs-function"><span class="hljs-params">(tag, value, {offset: offsetInChunk = <span class="hljs-number">0</span>, length = value.length, origin, generated, indentSize} = {})</span> -&gt;</span>
    token = [tag, value, @makeLocationData {offsetInChunk, length}]
    token.origin = origin <span class="hljs-keyword">if</span> origin
    token.generated = <span class="hljs-literal">yes</span> <span class="hljs-keyword">if</span> generated
    token.indentSize = indentSize <span class="hljs-keyword">if</span> indentSize?
    token</pre></div></div>
            
        </li>
        
        
        <li id="section-74">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-74">&#x00a7;</a>
              </div>
              <p>Add a token to the results.
<code>offset</code> is the offset into the current <code>@chunk</code> where the token starts.
<code>length</code> is the length of the token in the <code>@chunk</code>, after the offset.  If
not specified, the length of <code>value</code> will be used.</p>
<p>Returns the new token.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>  token: <span class="hljs-function"><span class="hljs-params">(tag, value, {offset, length, origin, data, generated, indentSize} = {})</span> -&gt;</span>
    token = @makeToken tag, value, {offset, length, origin, generated, indentSize}
    addTokenData token, data <span class="hljs-keyword">if</span> data
    @tokens.push token
    token</pre></div></div>
            
        </li>
        
        
        <li id="section-75">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-75">&#x00a7;</a>
              </div>
              <p>Peek at the last tag in the token stream.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>  tag: <span class="hljs-function">-&gt;</span>
    [..., token] = @tokens
    token?[<span class="hljs-number">0</span>]</pre></div></div>
            
        </li>
        
        
        <li id="section-76">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-76">&#x00a7;</a>
              </div>
              <p>Peek at the last value in the token stream.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>  value: <span class="hljs-function"><span class="hljs-params">(useOrigin = <span class="hljs-literal">no</span>)</span> -&gt;</span>
    [..., token] = @tokens
    <span class="hljs-keyword">if</span> useOrigin <span class="hljs-keyword">and</span> token?.origin?
      token.origin[<span class="hljs-number">1</span>]
    <span class="hljs-keyword">else</span>
      token?[<span class="hljs-number">1</span>]</pre></div></div>
            
        </li>
        
        
        <li id="section-77">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-77">&#x00a7;</a>
              </div>
              <p>Get the previous token in the token stream.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>  prev: <span class="hljs-function">-&gt;</span>
    @tokens[@tokens.length - <span class="hljs-number">1</span>]</pre></div></div>
            
        </li>
        
        
        <li id="section-78">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-78">&#x00a7;</a>
              </div>
              <p>Are we in the midst of an unfinished expression?</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>  unfinished: <span class="hljs-function">-&gt;</span>
    LINE_CONTINUER.test(@chunk) <span class="hljs-keyword">or</span>
    @tag() <span class="hljs-keyword">in</span> UNFINISHED

  validateUnicodeCodePointEscapes: <span class="hljs-function"><span class="hljs-params">(str, options)</span> -&gt;</span>
    replaceUnicodeCodePointEscapes str, merge options, {@error}</pre></div></div>
            
        </li>
        
        
        <li id="section-79">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-79">&#x00a7;</a>
              </div>
              <p>Validates escapes in strings and regexes.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>  validateEscapes: <span class="hljs-function"><span class="hljs-params">(str, options = {})</span> -&gt;</span>
    invalidEscapeRegex =
      <span class="hljs-keyword">if</span> options.isRegex
        REGEX_INVALID_ESCAPE
      <span class="hljs-keyword">else</span>
        STRING_INVALID_ESCAPE
    match = invalidEscapeRegex.exec str
    <span class="hljs-keyword">return</span> <span class="hljs-keyword">unless</span> match
    [[], before, octal, hex, unicodeCodePoint, unicode] = match
    message =
      <span class="hljs-keyword">if</span> octal
        <span class="hljs-string">&quot;octal escape sequences are not allowed&quot;</span>
      <span class="hljs-keyword">else</span>
        <span class="hljs-string">&quot;invalid escape sequence&quot;</span>
    invalidEscape = <span class="hljs-string">&quot;\\<span class="hljs-subst">#{octal <span class="hljs-keyword">or</span> hex <span class="hljs-keyword">or</span> unicodeCodePoint <span class="hljs-keyword">or</span> unicode}</span>&quot;</span>
    @error <span class="hljs-string">&quot;<span class="hljs-subst">#{message}</span> <span class="hljs-subst">#{invalidEscape}</span>&quot;</span>,
      offset: (options.offsetInChunk ? <span class="hljs-number">0</span>) + match.index + before.length
      length: invalidEscape.length

  suppressSemicolons: <span class="hljs-function">-&gt;</span>
    <span class="hljs-keyword">while</span> @value() <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;;&#x27;</span>
      @tokens.pop()
      @error <span class="hljs-string">&#x27;unexpected ;&#x27;</span> <span class="hljs-keyword">if</span> @prev()?[<span class="hljs-number">0</span>] <span class="hljs-keyword">in</span> [<span class="hljs-string">&#x27;=&#x27;</span>, UNFINISHED...]</pre></div></div>
            
        </li>
        
        
        <li id="section-80">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-80">&#x00a7;</a>
              </div>
              <p>Throws an error at either a given offset from the current chunk or at the
location of a token (<code>token[2]</code>).</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>  error: <span class="hljs-function"><span class="hljs-params">(message, options = {})</span> =&gt;</span>
    location =
      <span class="hljs-keyword">if</span> <span class="hljs-string">&#x27;first_line&#x27;</span> <span class="hljs-keyword">of</span> options
        options
      <span class="hljs-keyword">else</span>
        [first_line, first_column] = @getLineAndColumnFromChunk options.offset ? <span class="hljs-number">0</span>
        {first_line, first_column, last_column: first_column + (options.length ? <span class="hljs-number">1</span>) - <span class="hljs-number">1</span>}
    throwSyntaxError message, location</pre></div></div>
            
        </li>
        
        
        <li id="section-81">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-81">&#x00a7;</a>
              </div>
              <h2 id="helper-functions">Helper functions</h2>

            </div>
            
        </li>
        
        
        <li id="section-82">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-82">&#x00a7;</a>
              </div>
              
            </div>
            
            <div class="content"><div class='highlight'><pre><span class="hljs-function">
<span class="hljs-title">isUnassignable</span> = <span class="hljs-params">(name, displayName = name)</span> -&gt;</span> <span class="hljs-keyword">switch</span>
  <span class="hljs-keyword">when</span> name <span class="hljs-keyword">in</span> [JS_KEYWORDS..., COFFEE_KEYWORDS...]
    <span class="hljs-string">&quot;keyword &#x27;<span class="hljs-subst">#{displayName}</span>&#x27; can&#x27;t be assigned&quot;</span>
  <span class="hljs-keyword">when</span> name <span class="hljs-keyword">in</span> STRICT_PROSCRIBED
    <span class="hljs-string">&quot;&#x27;<span class="hljs-subst">#{displayName}</span>&#x27; can&#x27;t be assigned&quot;</span>
  <span class="hljs-keyword">when</span> name <span class="hljs-keyword">in</span> RESERVED
    <span class="hljs-string">&quot;reserved word &#x27;<span class="hljs-subst">#{displayName}</span>&#x27; can&#x27;t be assigned&quot;</span>
  <span class="hljs-keyword">else</span>
    <span class="hljs-literal">false</span>

<span class="hljs-built_in">exports</span>.isUnassignable = isUnassignable</pre></div></div>
            
        </li>
        
        
        <li id="section-83">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-83">&#x00a7;</a>
              </div>
              <p><code>from</code> isn’t a CoffeeScript keyword, but it behaves like one in <code>import</code> and
<code>export</code> statements (handled above) and in the declaration line of a <code>for</code>
loop. Try to detect when <code>from</code> is a variable identifier and when it is this
“sometimes” keyword.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre><span class="hljs-function"><span class="hljs-title">isForFrom</span> = <span class="hljs-params">(prev)</span> -&gt;</span></pre></div></div>
            
        </li>
        
        
        <li id="section-84">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-84">&#x00a7;</a>
              </div>
              <p><code>for i from iterable</code></p>

            </div>
            
            <div class="content"><div class='highlight'><pre>  <span class="hljs-keyword">if</span> prev[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;IDENTIFIER&#x27;</span>
    <span class="hljs-literal">yes</span></pre></div></div>
            
        </li>
        
        
        <li id="section-85">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-85">&#x00a7;</a>
              </div>
              <p><code>for from…</code></p>

            </div>
            
            <div class="content"><div class='highlight'><pre>  <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> prev[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">&#x27;FOR&#x27;</span>
    <span class="hljs-literal">no</span></pre></div></div>
            
        </li>
        
        
        <li id="section-86">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-86">&#x00a7;</a>
              </div>
              <p><code>for {from}…</code>, <code>for [from]…</code>, <code>for {a, from}…</code>, <code>for {a: from}…</code></p>

            </div>
            
            <div class="content"><div class='highlight'><pre>  <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> prev[<span class="hljs-number">1</span>] <span class="hljs-keyword">in</span> [<span class="hljs-string">&#x27;{&#x27;</span>, <span class="hljs-string">&#x27;[&#x27;</span>, <span class="hljs-string">&#x27;,&#x27;</span>, <span class="hljs-string">&#x27;:&#x27;</span>]
    <span class="hljs-literal">no</span>
  <span class="hljs-keyword">else</span>
    <span class="hljs-literal">yes</span>
<span class="hljs-function">
<span class="hljs-title">addTokenData</span> = <span class="hljs-params">(token, data)</span> -&gt;</span>
  <span class="hljs-built_in">Object</span>.assign (token.data ?= {}), data</pre></div></div>
            
        </li>
        
        
        <li id="section-87">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-87">&#x00a7;</a>
              </div>
              <h2 id="constants">Constants</h2>

            </div>
            
        </li>
        
        
        <li id="section-88">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-88">&#x00a7;</a>
              </div>
              
            </div>
            
        </li>
        
        
        <li id="section-89">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-89">&#x00a7;</a>
              </div>
              <p>Keywords that CoffeeScript shares in common with JavaScript.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>JS_KEYWORDS = [
  <span class="hljs-string">&#x27;true&#x27;</span>, <span class="hljs-string">&#x27;false&#x27;</span>, <span class="hljs-string">&#x27;null&#x27;</span>, <span class="hljs-string">&#x27;this&#x27;</span>
  <span class="hljs-string">&#x27;new&#x27;</span>, <span class="hljs-string">&#x27;delete&#x27;</span>, <span class="hljs-string">&#x27;typeof&#x27;</span>, <span class="hljs-string">&#x27;in&#x27;</span>, <span class="hljs-string">&#x27;instanceof&#x27;</span>
  <span class="hljs-string">&#x27;return&#x27;</span>, <span class="hljs-string">&#x27;throw&#x27;</span>, <span class="hljs-string">&#x27;break&#x27;</span>, <span class="hljs-string">&#x27;continue&#x27;</span>, <span class="hljs-string">&#x27;debugger&#x27;</span>, <span class="hljs-string">&#x27;yield&#x27;</span>, <span class="hljs-string">&#x27;await&#x27;</span>
  <span class="hljs-string">&#x27;if&#x27;</span>, <span class="hljs-string">&#x27;else&#x27;</span>, <span class="hljs-string">&#x27;switch&#x27;</span>, <span class="hljs-string">&#x27;for&#x27;</span>, <span class="hljs-string">&#x27;while&#x27;</span>, <span class="hljs-string">&#x27;do&#x27;</span>, <span class="hljs-string">&#x27;try&#x27;</span>, <span class="hljs-string">&#x27;catch&#x27;</span>, <span class="hljs-string">&#x27;finally&#x27;</span>
  <span class="hljs-string">&#x27;class&#x27;</span>, <span class="hljs-string">&#x27;extends&#x27;</span>, <span class="hljs-string">&#x27;super&#x27;</span>
  <span class="hljs-string">&#x27;import&#x27;</span>, <span class="hljs-string">&#x27;export&#x27;</span>, <span class="hljs-string">&#x27;default&#x27;</span>
]</pre></div></div>
            
        </li>
        
        
        <li id="section-90">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-90">&#x00a7;</a>
              </div>
              <p>CoffeeScript-only keywords.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>COFFEE_KEYWORDS = [
  <span class="hljs-string">&#x27;undefined&#x27;</span>, <span class="hljs-string">&#x27;Infinity&#x27;</span>, <span class="hljs-string">&#x27;NaN&#x27;</span>
  <span class="hljs-string">&#x27;then&#x27;</span>, <span class="hljs-string">&#x27;unless&#x27;</span>, <span class="hljs-string">&#x27;until&#x27;</span>, <span class="hljs-string">&#x27;loop&#x27;</span>, <span class="hljs-string">&#x27;of&#x27;</span>, <span class="hljs-string">&#x27;by&#x27;</span>, <span class="hljs-string">&#x27;when&#x27;</span>
]

COFFEE_ALIAS_MAP =
  <span class="hljs-keyword">and</span>  : <span class="hljs-string">&#x27;&amp;&amp;&#x27;</span>
  <span class="hljs-keyword">or</span>   : <span class="hljs-string">&#x27;||&#x27;</span>
  <span class="hljs-keyword">is</span>   : <span class="hljs-string">&#x27;==&#x27;</span>
  <span class="hljs-keyword">isnt</span> : <span class="hljs-string">&#x27;!=&#x27;</span>
  <span class="hljs-keyword">not</span>  : <span class="hljs-string">&#x27;!&#x27;</span>
  <span class="hljs-literal">yes</span>  : <span class="hljs-string">&#x27;true&#x27;</span>
  <span class="hljs-literal">no</span>   : <span class="hljs-string">&#x27;false&#x27;</span>
  <span class="hljs-literal">on</span>   : <span class="hljs-string">&#x27;true&#x27;</span>
  <span class="hljs-literal">off</span>  : <span class="hljs-string">&#x27;false&#x27;</span>

COFFEE_ALIASES  = (key <span class="hljs-keyword">for</span> key <span class="hljs-keyword">of</span> COFFEE_ALIAS_MAP)
COFFEE_KEYWORDS = COFFEE_KEYWORDS.concat COFFEE_ALIASES</pre></div></div>
            
        </li>
        
        
        <li id="section-91">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-91">&#x00a7;</a>
              </div>
              <p>The list of keywords that are reserved by JavaScript, but not used, or are
used by CoffeeScript internally. We throw an error when these are encountered,
to avoid having a JavaScript error at runtime.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>RESERVED = [
  <span class="hljs-string">&#x27;case&#x27;</span>, <span class="hljs-string">&#x27;function&#x27;</span>, <span class="hljs-string">&#x27;var&#x27;</span>, <span class="hljs-string">&#x27;void&#x27;</span>, <span class="hljs-string">&#x27;with&#x27;</span>, <span class="hljs-string">&#x27;const&#x27;</span>, <span class="hljs-string">&#x27;let&#x27;</span>, <span class="hljs-string">&#x27;enum&#x27;</span>
  <span class="hljs-string">&#x27;native&#x27;</span>, <span class="hljs-string">&#x27;implements&#x27;</span>, <span class="hljs-string">&#x27;interface&#x27;</span>, <span class="hljs-string">&#x27;package&#x27;</span>, <span class="hljs-string">&#x27;private&#x27;</span>
  <span class="hljs-string">&#x27;protected&#x27;</span>, <span class="hljs-string">&#x27;public&#x27;</span>, <span class="hljs-string">&#x27;static&#x27;</span>
]

STRICT_PROSCRIBED = [<span class="hljs-string">&#x27;arguments&#x27;</span>, <span class="hljs-string">&#x27;eval&#x27;</span>]</pre></div></div>
            
        </li>
        
        
        <li id="section-92">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-92">&#x00a7;</a>
              </div>
              <p>The superset of both JavaScript keywords and reserved words, none of which may
be used as identifiers or properties.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre><span class="hljs-built_in">exports</span>.JS_FORBIDDEN = JS_KEYWORDS.concat(RESERVED).concat(STRICT_PROSCRIBED)</pre></div></div>
            
        </li>
        
        
        <li id="section-93">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-93">&#x00a7;</a>
              </div>
              <p>The character code of the nasty Microsoft madness otherwise known as the BOM.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>BOM = <span class="hljs-number">65279</span></pre></div></div>
            
        </li>
        
        
        <li id="section-94">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-94">&#x00a7;</a>
              </div>
              <p>Token matching regexes.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>IDENTIFIER = <span class="hljs-regexp">/// ^
  (?!\d)
  ( (?: (?!\s)[$\w\x7f-\uffff] )+ )
  ( [^\n\S]* : (?!:) )?  <span class="hljs-comment"># Is this a property name?</span>
///</span></pre></div></div>
            
        </li>
        
        
        <li id="section-95">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-95">&#x00a7;</a>
              </div>
              <p>Like <code>IDENTIFIER</code>, but includes <code>-</code>s</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>JSX_IDENTIFIER_PART = <span class="hljs-regexp">/// (?: (?!\s)[\-$\w\x7f-\uffff] )+ ///</span>.source</pre></div></div>
            
        </li>
        
        
        <li id="section-96">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-96">&#x00a7;</a>
              </div>
              <p>In <a href="https://facebook.github.io/jsx/">https://facebook.github.io/jsx/</a> spec, JSXElementName can be
JSXIdentifier, JSXNamespacedName (JSXIdentifier : JSXIdentifier), or
JSXMemberExpression (two or more JSXIdentifier connected by <code>.</code>s).</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>JSX_IDENTIFIER = <span class="hljs-regexp">/// ^
  (?![\d&lt;]) <span class="hljs-comment"># Must not start with `&lt;`.</span>
  ( <span class="hljs-subst">#{JSX_IDENTIFIER_PART}</span>
    (?: \s* : \s* <span class="hljs-subst">#{JSX_IDENTIFIER_PART}</span>       <span class="hljs-comment"># JSXNamespacedName</span>
    | (?: \s* \. \s* <span class="hljs-subst">#{JSX_IDENTIFIER_PART}</span> )+ <span class="hljs-comment"># JSXMemberExpression</span>
    )? )
///</span></pre></div></div>
            
        </li>
        
        
        <li id="section-97">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-97">&#x00a7;</a>
              </div>
              <p>Fragment: &lt;&gt;&lt;/&gt;</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>JSX_FRAGMENT_IDENTIFIER = <span class="hljs-regexp">/// ^
  ()&gt; <span class="hljs-comment"># Ends immediately with `&gt;`.</span>
///</span></pre></div></div>
            
        </li>
        
        
        <li id="section-98">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-98">&#x00a7;</a>
              </div>
              <p>In <a href="https://facebook.github.io/jsx/">https://facebook.github.io/jsx/</a> spec, JSXAttributeName can be either
JSXIdentifier or JSXNamespacedName which is JSXIdentifier : JSXIdentifier</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>JSX_ATTRIBUTE = <span class="hljs-regexp">/// ^
  (?!\d)
  ( <span class="hljs-subst">#{JSX_IDENTIFIER_PART}</span>
    (?: \s* : \s* <span class="hljs-subst">#{JSX_IDENTIFIER_PART}</span>       <span class="hljs-comment"># JSXNamespacedName</span>
    )? )
  ( [^\S]* = (?!=) )?  <span class="hljs-comment"># Is this an attribute with a value?</span>
///</span>

NUMBER     = <span class="hljs-regexp">///
  ^ 0b[01](?:_?[01])*n?                         | <span class="hljs-comment"># binary</span>
  ^ 0o[0-7](?:_?[0-7])*n?                       | <span class="hljs-comment"># octal</span>
  ^ 0x[\da-f](?:_?[\da-f])*n?                   | <span class="hljs-comment"># hex</span>
  ^ \d+(?:_\d+)*n                               | <span class="hljs-comment"># decimal bigint</span>
  ^ (?:\d+(?:_\d+)*)?      \.? \d+(?:_\d+)*       <span class="hljs-comment"># decimal</span>
                     (?:e[+-]? \d+(?:_\d+)* )?
</span></pre></div></div>
            
        </li>
        
        
        <li id="section-99">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-99">&#x00a7;</a>
              </div>
              <p>decimal without support for numeric literal separators for reference:
\d*.?\d+ (?:e[+-]?\d+)?</p>

            </div>
            
            <div class="content"><div class='highlight'><pre><span class="hljs-regexp">///i

OPERATOR   = ///</span> ^ (
  ?: [-=]&gt;             <span class="hljs-comment"># function</span>
   | [-+*<span class="hljs-regexp">/%&lt;&gt;&amp;|^!?=]=  # compound assign /</span> compare
   | &gt;&gt;&gt;=?             <span class="hljs-comment"># zero-fill right shift</span>
   | ([-+:])\<span class="hljs-number">1</span>         <span class="hljs-comment"># doubles</span>
   | ([&amp;|&lt;&gt;*<span class="hljs-regexp">/%])\2=?   # logic /</span> shift / power / floor division / modulo
   | \?(\.|::)         <span class="hljs-comment"># soak access</span>
   | \.{<span class="hljs-number">2</span>,<span class="hljs-number">3</span>}           <span class="hljs-comment"># range or splat</span>
) <span class="hljs-regexp">///

WHITESPACE = /^[^\n\S]+/

COMMENT    = /^(\s*)<span class="hljs-comment">###([^#][\s\S]*?)(?:###([^\n\S]*)|###$)|^((?:\s*#(?!##[^#]).*)+)/</span>

CODE       = /^[-=]&gt;/

MULTI_DENT = /^(?:\n[^\n\S]*)+/

JSTOKEN      = ///</span>^ `<span class="language-javascript">(?!</span>``<span class="language-javascript">) ((?: [^</span>`\\] | \\[\s\S]           )*) `<span class="language-javascript">   <span class="hljs-comment">///</span>
<span class="hljs-variable constant_">HERE_JSTOKEN</span> = <span class="hljs-comment">///^ </span></span>```     ((?: [^`<span class="language-javascript">\\] | \\[\s\S] | </span>`(?!``) )*) ```<span class="language-javascript"> <span class="hljs-comment">///</span>

</span></pre></div></div>
            
        </li>
        
        
        <li id="section-100">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-100">&#x00a7;</a>
              </div>
              <p>String-matching-regexes.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>STRING_START   = <span class="hljs-regexp">/^(?:&#x27;&#x27;&#x27;|&quot;&quot;&quot;|&#x27;|&quot;)/</span>

STRING_SINGLE  = <span class="hljs-regexp">/// ^(?: [^\\&#x27;]  | \\[\s\S]                      )* ///</span>
STRING_DOUBLE  = <span class="hljs-regexp">/// ^(?: [^\\&quot;<span class="hljs-comment">#] | \\[\s\S] |           \#(?!\{) )* ///</span>
HEREDOC_SINGLE = ///</span> ^(?: [^\\<span class="hljs-string">&#x27;]  | \\[\s\S] | &#x27;</span>(?!<span class="hljs-string">&#x27;&#x27;</span>)            )* <span class="hljs-regexp">///
HEREDOC_DOUBLE = ///</span> ^(?: [^\\<span class="hljs-string">&quot;#] | \\[\s\S] | &quot;</span>(?!<span class="hljs-string">&quot;&quot;</span>) | \<span class="hljs-comment">#(?!\{) )* ///</span>

INSIDE_JSX = <span class="hljs-regexp">/// ^(?:
    [^
      \{ <span class="hljs-comment"># Start of CoffeeScript interpolation.</span>
      &lt;  <span class="hljs-comment"># Maybe JSX tag (`&lt;` not allowed even if bare).</span>
    ]
  )* ///</span> <span class="hljs-comment"># Similar to `HEREDOC_DOUBLE` but there is no escaping.</span>
JSX_INTERPOLATION = <span class="hljs-regexp">/// ^(?:
      \{       <span class="hljs-comment"># CoffeeScript interpolation.</span>
    | &lt;(?!/)   <span class="hljs-comment"># JSX opening tag.</span>
  )///</span>

HEREDOC_INDENT     = <span class="hljs-regexp">/\n+([^\n\S]*)(?=\S)/g</span></pre></div></div>
            
        </li>
        
        
        <li id="section-101">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-101">&#x00a7;</a>
              </div>
              <p>Regex-matching-regexes.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>REGEX = <span class="hljs-regexp">/// ^
  / (?!/) ((
  ?: [^ [ / \n \\ ]  <span class="hljs-comment"># Every other thing.</span>
   | \\[^\n]         <span class="hljs-comment"># Anything but newlines escaped.</span>
   | \[              <span class="hljs-comment"># Character class.</span>
       (?: \\[^\n] | [^ \] \n \\ ] )*
     \]
  )*) (/)?
///</span>

REGEX_FLAGS  = <span class="hljs-regexp">/^\w*/</span>
VALID_FLAGS  = <span class="hljs-regexp">/^(?!.*(.).*\1)[gimsuy]*$/</span>

HEREGEX      = <span class="hljs-regexp">/// ^
  (?:
</span></pre></div></div>
            
        </li>
        
        
        <li id="section-102">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-102">&#x00a7;</a>
              </div>
              <p>Match any character, except those that need special handling below.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>      [^\\/<span class="hljs-comment">#\s]</span></pre></div></div>
            
        </li>
        
        
        <li id="section-103">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-103">&#x00a7;</a>
              </div>
              <p>Match <code>\</code> followed by any character.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>    | \\[\s\S]</pre></div></div>
            
        </li>
        
        
        <li id="section-104">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-104">&#x00a7;</a>
              </div>
              <p>Match any <code>/</code> except <code>///</code>.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>    | <span class="hljs-regexp">/(?!/</span>/)</pre></div></div>
            
        </li>
        
        
        <li id="section-105">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-105">&#x00a7;</a>
              </div>
              <p>Match <code>#</code> which is not part of interpolation, e.g. <code>#{}</code>.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>    | \<span class="hljs-comment">#(?!\{)</span></pre></div></div>
            
        </li>
        
        
        <li id="section-106">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-106">&#x00a7;</a>
              </div>
              <p>Comments consume everything until the end of the line, including <code>///</code>.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>    | \s+(?:<span class="hljs-comment">#(?!\{).*)?</span>
  )*
<span class="hljs-regexp">///

HEREGEX_COMMENT = /(\s+)(<span class="hljs-comment">#(?!{).*)/gm</span>

REGEX_ILLEGAL = ///</span> ^ ( / | <span class="hljs-regexp">/{3}\s*) (\*) /</span><span class="hljs-regexp">//</span>

POSSIBLY_DIVISION   = <span class="hljs-regexp">/// ^ /=?\s ///</span></pre></div></div>
            
        </li>
        
        
        <li id="section-107">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-107">&#x00a7;</a>
              </div>
              <p>Other regexes.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>HERECOMMENT_ILLEGAL = <span class="hljs-regexp">/\*\//</span>

LINE_CONTINUER      = <span class="hljs-regexp">/// ^ \s* (?: , | \??\.(?![.\d]) | \??:: ) ///</span>

STRING_INVALID_ESCAPE = <span class="hljs-regexp">///
  ( (?:^|[^\\]) (?:\\\\)* )        <span class="hljs-comment"># Make sure the escape isn’t escaped.</span>
  \\ (
     ?: (0\d|[1-7])                <span class="hljs-comment"># octal escape</span>
      | (x(?![\da-fA-F]{2}).{0,2}) <span class="hljs-comment"># hex escape</span>
      | (u\{(?![\da-fA-F]{1,}\})[^}]*\}?) <span class="hljs-comment"># unicode code point escape</span>
      | (u(?!\{|[\da-fA-F]{4}).{0,4}) <span class="hljs-comment"># unicode escape</span>
  )
///</span>
REGEX_INVALID_ESCAPE = <span class="hljs-regexp">///
  ( (?:^|[^\\]) (?:\\\\)* )        <span class="hljs-comment"># Make sure the escape isn’t escaped.</span>
  \\ (
     ?: (0\d)                      <span class="hljs-comment"># octal escape</span>
      | (x(?![\da-fA-F]{2}).{0,2}) <span class="hljs-comment"># hex escape</span>
      | (u\{(?![\da-fA-F]{1,}\})[^}]*\}?) <span class="hljs-comment"># unicode code point escape</span>
      | (u(?!\{|[\da-fA-F]{4}).{0,4}) <span class="hljs-comment"># unicode escape</span>
  )
///</span>

TRAILING_SPACES     = <span class="hljs-regexp">/\s+$/</span></pre></div></div>
            
        </li>
        
        
        <li id="section-108">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-108">&#x00a7;</a>
              </div>
              <p>Compound assignment tokens.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>COMPOUND_ASSIGN = [
  <span class="hljs-string">&#x27;-=&#x27;</span>, <span class="hljs-string">&#x27;+=&#x27;</span>, <span class="hljs-string">&#x27;/=&#x27;</span>, <span class="hljs-string">&#x27;*=&#x27;</span>, <span class="hljs-string">&#x27;%=&#x27;</span>, <span class="hljs-string">&#x27;||=&#x27;</span>, <span class="hljs-string">&#x27;&amp;&amp;=&#x27;</span>, <span class="hljs-string">&#x27;?=&#x27;</span>, <span class="hljs-string">&#x27;&lt;&lt;=&#x27;</span>, <span class="hljs-string">&#x27;&gt;&gt;=&#x27;</span>, <span class="hljs-string">&#x27;&gt;&gt;&gt;=&#x27;</span>
  <span class="hljs-string">&#x27;&amp;=&#x27;</span>, <span class="hljs-string">&#x27;^=&#x27;</span>, <span class="hljs-string">&#x27;|=&#x27;</span>, <span class="hljs-string">&#x27;**=&#x27;</span>, <span class="hljs-string">&#x27;//=&#x27;</span>, <span class="hljs-string">&#x27;%%=&#x27;</span>
]</pre></div></div>
            
        </li>
        
        
        <li id="section-109">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-109">&#x00a7;</a>
              </div>
              <p>Unary tokens.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>UNARY = [<span class="hljs-string">&#x27;NEW&#x27;</span>, <span class="hljs-string">&#x27;TYPEOF&#x27;</span>, <span class="hljs-string">&#x27;DELETE&#x27;</span>]

UNARY_MATH = [<span class="hljs-string">&#x27;!&#x27;</span>, <span class="hljs-string">&#x27;~&#x27;</span>]</pre></div></div>
            
        </li>
        
        
        <li id="section-110">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-110">&#x00a7;</a>
              </div>
              <p>Bit-shifting tokens.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>SHIFT = [<span class="hljs-string">&#x27;&lt;&lt;&#x27;</span>, <span class="hljs-string">&#x27;&gt;&gt;&#x27;</span>, <span class="hljs-string">&#x27;&gt;&gt;&gt;&#x27;</span>]</pre></div></div>
            
        </li>
        
        
        <li id="section-111">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-111">&#x00a7;</a>
              </div>
              <p>Comparison tokens.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>COMPARE = [<span class="hljs-string">&#x27;==&#x27;</span>, <span class="hljs-string">&#x27;!=&#x27;</span>, <span class="hljs-string">&#x27;&lt;&#x27;</span>, <span class="hljs-string">&#x27;&gt;&#x27;</span>, <span class="hljs-string">&#x27;&lt;=&#x27;</span>, <span class="hljs-string">&#x27;&gt;=&#x27;</span>]</pre></div></div>
            
        </li>
        
        
        <li id="section-112">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-112">&#x00a7;</a>
              </div>
              <p>Mathematical tokens.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>MATH = [<span class="hljs-string">&#x27;*&#x27;</span>, <span class="hljs-string">&#x27;/&#x27;</span>, <span class="hljs-string">&#x27;%&#x27;</span>, <span class="hljs-string">&#x27;//&#x27;</span>, <span class="hljs-string">&#x27;%%&#x27;</span>]</pre></div></div>
            
        </li>
        
        
        <li id="section-113">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-113">&#x00a7;</a>
              </div>
              <p>Relational tokens that are negatable with <code>not</code> prefix.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>RELATION = [<span class="hljs-string">&#x27;IN&#x27;</span>, <span class="hljs-string">&#x27;OF&#x27;</span>, <span class="hljs-string">&#x27;INSTANCEOF&#x27;</span>]</pre></div></div>
            
        </li>
        
        
        <li id="section-114">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-114">&#x00a7;</a>
              </div>
              <p>Boolean tokens.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>BOOL = [<span class="hljs-string">&#x27;TRUE&#x27;</span>, <span class="hljs-string">&#x27;FALSE&#x27;</span>]</pre></div></div>
            
        </li>
        
        
        <li id="section-115">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-115">&#x00a7;</a>
              </div>
              <p>Tokens which could legitimately be invoked or indexed. An opening
parentheses or bracket following these tokens will be recorded as the start
of a function invocation or indexing operation.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>CALLABLE  = [<span class="hljs-string">&#x27;IDENTIFIER&#x27;</span>, <span class="hljs-string">&#x27;PROPERTY&#x27;</span>, <span class="hljs-string">&#x27;)&#x27;</span>, <span class="hljs-string">&#x27;]&#x27;</span>, <span class="hljs-string">&#x27;?&#x27;</span>, <span class="hljs-string">&#x27;@&#x27;</span>, <span class="hljs-string">&#x27;THIS&#x27;</span>, <span class="hljs-string">&#x27;SUPER&#x27;</span>, <span class="hljs-string">&#x27;DYNAMIC_IMPORT&#x27;</span>]
INDEXABLE = CALLABLE.concat [
  <span class="hljs-string">&#x27;NUMBER&#x27;</span>, <span class="hljs-string">&#x27;INFINITY&#x27;</span>, <span class="hljs-string">&#x27;NAN&#x27;</span>, <span class="hljs-string">&#x27;STRING&#x27;</span>, <span class="hljs-string">&#x27;STRING_END&#x27;</span>, <span class="hljs-string">&#x27;REGEX&#x27;</span>, <span class="hljs-string">&#x27;REGEX_END&#x27;</span>
  <span class="hljs-string">&#x27;BOOL&#x27;</span>, <span class="hljs-string">&#x27;NULL&#x27;</span>, <span class="hljs-string">&#x27;UNDEFINED&#x27;</span>, <span class="hljs-string">&#x27;}&#x27;</span>, <span class="hljs-string">&#x27;::&#x27;</span>
]</pre></div></div>
            
        </li>
        
        
        <li id="section-116">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-116">&#x00a7;</a>
              </div>
              <p>Tokens which can be the left-hand side of a less-than comparison, i.e. <code>a&lt;b</code>.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>COMPARABLE_LEFT_SIDE = [<span class="hljs-string">&#x27;IDENTIFIER&#x27;</span>, <span class="hljs-string">&#x27;)&#x27;</span>, <span class="hljs-string">&#x27;]&#x27;</span>, <span class="hljs-string">&#x27;NUMBER&#x27;</span>]</pre></div></div>
            
        </li>
        
        
        <li id="section-117">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-117">&#x00a7;</a>
              </div>
              <p>Tokens which a regular expression will never immediately follow (except spaced
CALLABLEs in some cases), but which a division operator can.</p>
<p>See: <a href="http://www-archive.mozilla.org/js/language/js20-2002-04/rationale/syntax.html#regular-expressions">http://www-archive.mozilla.org/js/language/js20-2002-04/rationale/syntax.html#regular-expressions</a></p>

            </div>
            
            <div class="content"><div class='highlight'><pre>NOT_REGEX = INDEXABLE.concat [<span class="hljs-string">&#x27;++&#x27;</span>, <span class="hljs-string">&#x27;--&#x27;</span>]</pre></div></div>
            
        </li>
        
        
        <li id="section-118">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-118">&#x00a7;</a>
              </div>
              <p>Tokens that, when immediately preceding a <code>WHEN</code>, indicate that the <code>WHEN</code>
occurs at the start of a line. We disambiguate these from trailing whens to
avoid an ambiguity in the grammar.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>LINE_BREAK = [<span class="hljs-string">&#x27;INDENT&#x27;</span>, <span class="hljs-string">&#x27;OUTDENT&#x27;</span>, <span class="hljs-string">&#x27;TERMINATOR&#x27;</span>]</pre></div></div>
            
        </li>
        
        
        <li id="section-119">
            <div class="annotation">
              
              <div class="sswrap ">
                <a class="ss" href="#section-119">&#x00a7;</a>
              </div>
              <p>Additional indent in front of these is ignored.</p>

            </div>
            
            <div class="content"><div class='highlight'><pre>INDENTABLE_CLOSERS = [<span class="hljs-string">&#x27;)&#x27;</span>, <span class="hljs-string">&#x27;}&#x27;</span>, <span class="hljs-string">&#x27;]&#x27;</span>]</pre></div></div>
            
        </li>
        
    </ul>
  </div>
</body>
</html>
